Sep 14 07:49:04.179059 ip-10-0-135-172 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 07:49:04.559778 ip-10-0-135-172 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:04.559778 ip-10-0-135-172 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 07:49:04.559778 ip-10-0-135-172 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:04.559778 ip-10-0-135-172 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 07:49:04.559778 ip-10-0-135-172 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:49:04.560469 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.559843    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 07:49:04.563484 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563468    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:04.563484 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563483    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563487    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563490    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563494    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563496    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563500    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563503    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563506    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563509    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563511    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563514    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563517    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563519    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563522    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563525    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563527    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563530    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563533    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563542    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563545    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:04.563548 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563547    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563550    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563553    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563556    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563559    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563562    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563565    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563570    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563574    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563577    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563580    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563582    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563585    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563588    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563590    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563593    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563595    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563598    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563601    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563604    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:04.564042 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563606    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563609    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563611    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563613    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563616    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563619    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563621    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563624    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563627    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563629    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563632    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563635    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563637    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563640    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563644    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563647    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563649    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563652    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563654    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:04.564535 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563657    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563660    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563662    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563665    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563667    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563670    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563672    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563675    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563677    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563680    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563682    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563685    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563687    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563690    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563694    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563696    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563699    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563701    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563704    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563706    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:04.565026 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563709    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563712    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563714    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563717    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563719    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563733    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563736    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563739    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563744    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563747    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563750    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563754    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563757    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563760    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563763    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563766    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563770    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563774    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.563777    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:04.565505 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564452    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564459    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564462    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564465    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564468    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564471    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564473    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564476    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564479    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564482    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564484    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564487    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564489    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564492    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564495    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564499    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564503    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564506    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564510    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:04.565979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564513    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564517    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564521    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564523    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564526    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564529    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564531    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564534    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564536    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564539    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564542    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564545    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564548    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564550    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564553    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564555    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564558    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564560    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564563    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564565    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:04.566458 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564568    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564571    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564573    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564576    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564578    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564581    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564583    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564586    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564588    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564591    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564594    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564597    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564600    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564603    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564606    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564609    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564611    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564614    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564616    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564619    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:04.566960 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564622    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564625    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564628    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564630    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564633    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564635    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564638    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564641    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564643    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564646    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564648    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564652    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564655    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564658    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564661    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564663    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564666    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564669    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564672    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:04.567448 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564676    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564679    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564681    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564684    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564687    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564690    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564695    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564698    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564701    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564703    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564707    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564709    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564712    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564714    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564717    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564720    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564737    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564739    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564742    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:04.567917 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564745    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.564748    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565509    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565514    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565518    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565521    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565524    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565526    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565529    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565532    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565535    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565537    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565540    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565543    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565545    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565548    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565551    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565554    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565556    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:04.568397 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565560    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565563    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565566    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565569    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565571    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565574    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565576    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565580    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565584    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565587    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565591    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565594    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565599    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565602    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565605    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565608    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565611    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565614    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565616    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:04.568875 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565619    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565622    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565624    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565627    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565629    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565632    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565634    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565637    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565640    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565642    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565645    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565648    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565651    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565654    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565656    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565660    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565663    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565665    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565668    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565670    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:04.569356 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565673    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565675    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565678    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565680    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565683    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565686    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565688    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565690    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565693    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565697    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565699    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565701    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565704    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565707    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565710    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565712    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565715    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565717    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565720    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565737    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:04.569859 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565740    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565743    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565746    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565749    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565752    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565755    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565757    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565760    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565763    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565765    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565768    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565770    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565773    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565775    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565778    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565780    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565783    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565785    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565788    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565790    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:04.570374 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565793    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565795    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.565798    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565864    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565910    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565916    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565921    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565925    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565928    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565932    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565936    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565939    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565942    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565946    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565950    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565953    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565956    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565959    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565962    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565965    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565968    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565975    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565978    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 07:49:04.570866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565981    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565984    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565988    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565992    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565995    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.565998    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566002    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566005    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566008    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566011    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566015    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566018    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566022    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566025    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566028    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566031    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566035    2563 flags.go:64] FLAG: --enable-server="true"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566038    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566042    2563 flags.go:64] FLAG: --event-burst="100"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566045    2563 flags.go:64] FLAG: --event-qps="50"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566048    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566051    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566054    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566058    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 07:49:04.571417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566061    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566064    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566067    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566070    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566073    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566076    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566079    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566082    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566085    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566088    2563 flags.go:64] FLAG: --feature-gates=""
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566091    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566094    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566097    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566100    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566103    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566106    2563 flags.go:64] FLAG: --help="false"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566109    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566112    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566115    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566118    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566122    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566125    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566128    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 07:49:04.572051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566131    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566134    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566137    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566140    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566143    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566145    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566148    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566151    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566154    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566157    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566160    2563 flags.go:64] FLAG: --lock-file=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566162    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566165    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566168    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566173    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566176    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566179    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566186    2563 flags.go:64] FLAG: --logging-format="text"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566189    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566193    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566196    2563 flags.go:64] FLAG: --manifest-url=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566199    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566203    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566206    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566210    2563 flags.go:64] FLAG: --max-pods="110"
Sep 14 07:49:04.572741 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566213    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566216    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566221    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566224    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566227    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566230    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566233    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566240    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566243    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566247    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566250    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566253    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566258    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566260    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566263    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566266    2563 flags.go:64] FLAG: --port="10250"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566269    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566273    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01d9f952687023fec"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566276    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566279    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566282    2563 flags.go:64] FLAG: --register-node="true"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566285    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566289    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566292    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 14 07:49:04.573363 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566295    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566298    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566304    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566307    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566310    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566313    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566316    2563 flags.go:64] FLAG: --runonce="false"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566319    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566322    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566324    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566327    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566332    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566335    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566338    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566341    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566344    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566347    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566350    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566353    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566356    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566359    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566362    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566367    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566370    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566373    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 07:49:04.574035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566377    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566380    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566383    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566386    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566389    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566391    2563 flags.go:64] FLAG: --v="2"
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566395    2563 flags.go:64] FLAG: --version="false"
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566399    2563 flags.go:64] FLAG: --vmodule=""
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566403    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.566407    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566518    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566522    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566525    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566528    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566531    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566534    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566536    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566539    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566544    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566546    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566549    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:04.574990 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566551    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566556    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566559    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566561    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566564    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566567    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566570    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566573    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566576    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566578    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566581    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566584    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566586    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566589    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566592    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566594    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566599    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566603    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566606    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:04.575645 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566609    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566612    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566615    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566618    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566622    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566625    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566627    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566630    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566633    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566636    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566640    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566642    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566645    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566647    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566652    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566654    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566657    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566660    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566663    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566665    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:04.576179 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566668    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566671    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566673    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566677    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566681    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566684    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566687    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566689    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566692    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566695    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566697    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566700    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566703    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566705    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566708    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566711    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566714    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566717    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566719    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566734    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:04.576678 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566737    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566740    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566744    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566747    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566750    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566752    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566757    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566760    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566762    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566765    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566769    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566773    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566776    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566780    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566783    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566786    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566789    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566791    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566794    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566798    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:04.577189 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566800    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566804    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566807    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566810    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566813    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566816    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566819    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566821    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.566825    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.567416    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.576351    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 07:49:04.577687 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.576371    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.577747    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.577755    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577811    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577816    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577819    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577823    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577825    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577828    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577831    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577834    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577837    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577839    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577842    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577844    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577847    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577850    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577853    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577856    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577859    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:04.577983 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577861    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577864    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577867    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577869    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577872    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577874    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577877    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577880    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577882    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577885    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577887    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577890    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577893    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577895    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577899    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577902    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577905    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577908    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577910    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:04.578469 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577913    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577917    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577922    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577926    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577929    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577932    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577935    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577937    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577940    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577943    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577946    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577949    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577952    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577955    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577957    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577960    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577962    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577965    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577967    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:04.578949 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577970    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577972    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577975    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577977    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577980    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577983    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577985    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577988    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577992    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577995    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.577998    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578000    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578002    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578005    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578008    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578010    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578013    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578016    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578018    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:04.579438 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578021    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578023    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578026    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578028    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578031    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578039    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578042    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578044    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578047    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578051    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578055    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578058    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578060    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578063    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578065    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578068    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578070    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578073    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578076    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578079    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:04.579994 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578081    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578084    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578087    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578090    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578092    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.578097    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578202    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578207    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578210    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578213    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578216    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578219    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578222    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578225    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578228    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:49:04.580476 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578230    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578233    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578236    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578240    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578243    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578246    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578248    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578252    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578255    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578258    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578260    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578263    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578266    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578269    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578272    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578275    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578277    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578280    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578283    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578286    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:49:04.580897 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578288    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578291    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578294    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578297    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578299    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578302    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578305    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578309    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578313    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578316    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578319    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578322    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578324    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578327    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578330    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578332    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578335    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578337    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578340    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578343    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:49:04.581390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578345    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578348    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578350    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578353    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578356    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578358    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578361    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578363    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578367    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578371    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578374    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578377    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578380    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578384    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578387    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578389    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578392    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578395    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578397    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:49:04.581902 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578400    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578402    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578405    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578407    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578410    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578412    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578415    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578417    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578420    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578422    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578426    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578428    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578431    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578434    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578437    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578439    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578442    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578444    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578447    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578449    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:49:04.582366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578452    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578454    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578457    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578459    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578462    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578464    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578467    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578470    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578472    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578475    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:04.578478    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.578483    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.578572    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 07:49:04.582908 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.582315    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 07:49:04.583359 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.583346    2563 server.go:1019] "Starting client certificate rotation"
Sep 14 07:49:04.583460 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.583446    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:49:04.583510 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.583501    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:49:04.585958 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.585943    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:49:04.589395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.589377    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:49:04.598887 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.598863    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 14 07:49:04.602668 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.602654    2563 log.go:25] "Validated CRI v1 image API"
Sep 14 07:49:04.603489 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.603473    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 07:49:04.607541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.607522    2563 fs.go:135] Filesystem UUIDs: map[00b228c2-17a9-46b9-9b91-8eaa32f4eeaa:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7f5e0942-b323-4a94-ad28-7c81154a4cfd:/dev/nvme0n1p4]
Sep 14 07:49:04.607611 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.607540    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 07:49:04.612941 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.612921    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:49:04.613917 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.613814    2563 manager.go:216] Machine: {Timestamp:2026-09-14 07:49:04.612530567 +0000 UTC m=+0.320544697 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111257 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec255d0a771bdad708db0aa64f39e6b1 SystemUUID:ec255d0a-771b-dad7-08db-0aa64f39e6b1 BootID:102606e9-f425-452d-8956-78615bcfbbc9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e6:5a:f7:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e6:5a:f7:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:25:cc:7d:98:e2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 07:49:04.613917 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.613912    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 07:49:04.614027 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.613987    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 07:49:04.616826 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.616805    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 07:49:04.616950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.616831    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-172.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 07:49:04.617982 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.617972    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 07:49:04.618019 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.617985    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 07:49:04.618019 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.617993    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:49:04.618571 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.618561    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:49:04.619536 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.619526    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 07:49:04.622537 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622525    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:49:04.622649 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622640    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 07:49:04.622680 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622669    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 07:49:04.622680 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622677    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 07:49:04.622745 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622690    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 07:49:04.622745 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622700    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 14 07:49:04.622745 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.622708    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 07:49:04.623053 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.623042    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:49:04.623108 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.623062    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:49:04.625516 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.625502    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 07:49:04.627597 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.627580    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 07:49:04.627673 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.627622    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 07:49:04.629946 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.629931    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.629956    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.629966    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.629974    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.629986    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.629994    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630013    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 07:49:04.630020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630020    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 07:49:04.630238 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630029    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 07:49:04.630238 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630041    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 07:49:04.630238 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630064    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 07:49:04.630238 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630072    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 07:49:04.630763 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630751    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 07:49:04.630811 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.630765    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 07:49:04.632169 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.632144    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 07:49:04.632338 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.632311    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 07:49:04.632556 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.632542    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-172.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 07:49:04.634446 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.634431    2563 server.go:1268] "Started kubelet"
Sep 14 07:49:04.634516 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.634481    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 07:49:04.634572 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.634529    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 07:49:04.634622 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.634595    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 07:49:04.635171 ip-10-0-135-172 systemd[1]: Started Kubernetes Kubelet.
Sep 14 07:49:04.636420 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.636399    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 07:49:04.636876 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.636858    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 07:49:04.637071 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.637030    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zmszj"
Sep 14 07:49:04.639103 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.639085    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 07:49:04.640539 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.639648    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-172.ec2.internal.18d5207c2be9a277  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-172.ec2.internal,UID:ip-10-0-135-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-172.ec2.internal,},FirstTimestamp:2026-09-14 07:49:04.634405495 +0000 UTC m=+0.342419625,LastTimestamp:2026-09-14 07:49:04.634405495 +0000 UTC m=+0.342419625,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-172.ec2.internal,}"
Sep 14 07:49:04.641606 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.641593    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 07:49:04.641669 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.641640    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 07:49:04.641739 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.641701    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:04.642378 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.642357    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 07:49:04.642378 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.642362    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 07:49:04.642510 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.642386    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 07:49:04.642510 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.642472    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 07:49:04.642510 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.642478    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 07:49:04.643519 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.643488    2563 factory.go:55] Registering systemd factory
Sep 14 07:49:04.643519 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.643512    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 14 07:49:04.644051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.644035    2563 factory.go:153] Registering CRI-O factory
Sep 14 07:49:04.644051 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.644053    2563 factory.go:223] Registration of the crio container factory successfully
Sep 14 07:49:04.644175 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.644111    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 07:49:04.644175 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.644132    2563 factory.go:103] Registering Raw factory
Sep 14 07:49:04.644175 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.644153    2563 manager.go:1196] Started watching for new ooms in manager
Sep 14 07:49:04.644575 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.644559    2563 manager.go:349] Starting recovery of all containers
Sep 14 07:49:04.645314 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.645288    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zmszj"
Sep 14 07:49:04.653626 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.653606    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:04.655995 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.655821    2563 manager.go:354] Recovery completed
Sep 14 07:49:04.656572 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.656552    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-172.ec2.internal\" not found" node="ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.662665 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.662653    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.664215 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664198    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.664287 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664224    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.664287 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664240    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.664803 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664788    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 07:49:04.664803 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664801    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 07:49:04.664938 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664829    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.664938 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.664842    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:49:04.665942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.665929    2563 policy_none.go:49] "None policy: Start"
Sep 14 07:49:04.666001 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.665948    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 07:49:04.666001 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.665960    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 07:49:04.666286 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.666270    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.666347 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.666298    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.666347 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.666308    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.666953 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.666941    2563 policy_none.go:47] "Start"
Sep 14 07:49:04.691312 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691299    2563 manager.go:337] "Starting Device Plugin manager"
Sep 14 07:49:04.691406 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.691331    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 07:49:04.691406 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691341    2563 server.go:87] "Starting device plugin registration server"
Sep 14 07:49:04.691661 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691646    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 07:49:04.691766 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691662    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 07:49:04.691912 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691837    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 07:49:04.691912 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691908    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 07:49:04.692024 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.691916    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 07:49:04.692383 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.692358    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 07:49:04.692450 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.692405    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:04.780753 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.780712    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 07:49:04.781851 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.781829    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 07:49:04.781938 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.781872    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 07:49:04.781938 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.781892    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 07:49:04.782024 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.781927    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 07:49:04.784311 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.784295    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:04.792538 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.792525    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.793395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.793379    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.793479 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.793411    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.793479 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.793423    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.793479 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.793443    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.799315 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.799301    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.799408 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.799323    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-172.ec2.internal\": node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:04.823446 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.823375    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:04.883294 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.882644    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"]
Sep 14 07:49:04.883389 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.883331    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.884250 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.884236    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.884342 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.884267    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.884342 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.884282    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.885457 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.885442    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.885600 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.885585    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.885647 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.885614    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.886165 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.886150    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.886165 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.886161    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.886277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.886176    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.886277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.886185    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.886277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.886185    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.886389 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.886291    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.887310 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.887297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.887354 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.887320    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:49:04.887959 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.887945    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:49:04.887959 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.887968    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:49:04.888100 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.887983    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:49:04.912312 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.912290    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-172.ec2.internal\" not found" node="ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.916556 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.916539    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-172.ec2.internal\" not found" node="ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.923580 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:04.923566    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:04.943942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.943922    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c813448e0dd7de1c8f6d146bb680f0b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"c813448e0dd7de1c8f6d146bb680f0b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.944016 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.943945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c813448e0dd7de1c8f6d146bb680f0b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"c813448e0dd7de1c8f6d146bb680f0b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:04.944016 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:04.943972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2ac03057413443baa8a950301b2f270d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-172.ec2.internal\" (UID: \"2ac03057413443baa8a950301b2f270d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.024285 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.024255    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.044588 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.044569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c813448e0dd7de1c8f6d146bb680f0b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"c813448e0dd7de1c8f6d146bb680f0b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.044660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.044592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c813448e0dd7de1c8f6d146bb680f0b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"c813448e0dd7de1c8f6d146bb680f0b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.044660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.044608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2ac03057413443baa8a950301b2f270d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-172.ec2.internal\" (UID: \"2ac03057413443baa8a950301b2f270d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.044660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.044644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2ac03057413443baa8a950301b2f270d-config\") pod \"kube-apiserver-proxy-ip-10-0-135-172.ec2.internal\" (UID: \"2ac03057413443baa8a950301b2f270d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.044809 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.044682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c813448e0dd7de1c8f6d146bb680f0b3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"c813448e0dd7de1c8f6d146bb680f0b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.044809 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.044687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c813448e0dd7de1c8f6d146bb680f0b3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal\" (UID: \"c813448e0dd7de1c8f6d146bb680f0b3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.124953 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.124893    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.214394 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.214355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.219396 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.219234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.225857 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.225836    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.326228 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.326185    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.426341 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.426282    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.527315 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.527282    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.583058 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.583039    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 07:49:05.583479 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.583174    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:49:05.583479 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.583206    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:49:05.628087 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:05.628060    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-172.ec2.internal\" not found"
Sep 14 07:49:05.650094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.650065    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 07:44:05 +0000 UTC" deadline="2028-05-28 06:46:53.034482072 +0000 UTC"
Sep 14 07:49:05.650094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.650093    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14926h57m47.384393563s"
Sep 14 07:49:05.714529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.714500    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:05.742393 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.742365    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.757183 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.757164    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:49:05.758962 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.758940    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"
Sep 14 07:49:05.774046 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.774026    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:49:05.807869 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:05.807668    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc813448e0dd7de1c8f6d146bb680f0b3.slice/crio-72dcd73d63d61a486b760a4d71a9d0c93129e0d17fadec5ca620a3658c88540a WatchSource:0}: Error finding container 72dcd73d63d61a486b760a4d71a9d0c93129e0d17fadec5ca620a3658c88540a: Status 404 returned error can't find the container with id 72dcd73d63d61a486b760a4d71a9d0c93129e0d17fadec5ca620a3658c88540a
Sep 14 07:49:05.808159 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:05.808142    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac03057413443baa8a950301b2f270d.slice/crio-9a2a4baadd024f26c89a524e27f745a94cd68e22447b3a326195535dbd13839b WatchSource:0}: Error finding container 9a2a4baadd024f26c89a524e27f745a94cd68e22447b3a326195535dbd13839b: Status 404 returned error can't find the container with id 9a2a4baadd024f26c89a524e27f745a94cd68e22447b3a326195535dbd13839b
Sep 14 07:49:05.812466 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:05.812439    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:49:06.159573 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.159481    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:06.610371 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.610271    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:06.624606 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.624571    2563 apiserver.go:52] "Watching apiserver"
Sep 14 07:49:06.635641 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.635613    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 07:49:06.636114 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.636080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-j786z","openshift-multus/multus-additional-cni-plugins-5gctk","openshift-multus/multus-xtts5","openshift-network-operator/iptables-alerter-wj55g","openshift-ovn-kubernetes/ovnkube-node-hl96f","kube-system/global-pull-secret-syncer-cxxtl","kube-system/konnectivity-agent-xdppf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw","openshift-cluster-node-tuning-operator/tuned-72bzr","openshift-image-registry/node-ca-2jjqz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal","openshift-multus/network-metrics-daemon-7xmvb","openshift-network-diagnostics/network-check-target-s7jl4","kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal"]
Sep 14 07:49:06.637828 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.637703    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.639250 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.638949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:06.639250 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.639050    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:06.640395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.640372    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.641796 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.641736    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 07:49:06.642032 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.642013    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 07:49:06.645185 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.644731    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fr5zv\""
Sep 14 07:49:06.645185 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.644745    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 07:49:06.645185 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.645109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.646156 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.646084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.646883 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.646418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.646883 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.646515    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 07:49:06.647580 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.646421    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f4w6c\""
Sep 14 07:49:06.647580 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.647492    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.647725 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.647612    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:06.648245 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.648060    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.648245 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.648090    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.648403 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.648264    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 07:49:06.652754 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.652560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p5vq8\""
Sep 14 07:49:06.652754 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.652594    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 07:49:06.652891 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.652829    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.653310 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653291    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.653550 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-cni-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.653653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-conf-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.653653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-systemd\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.653653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-cni-netd\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.653847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653665    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.653847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-env-overrides\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.653847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653745    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 07:49:06.653847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be159616-10cd-4bbc-9db6-dccb069099cd-cni-binary-copy\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.653847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-socket-dir-parent\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.653847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg6wp\" (UniqueName: \"kubernetes.io/projected/be159616-10cd-4bbc-9db6-dccb069099cd-kube-api-access-sg6wp\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-log-socket\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/748bdf6e-1c14-4269-b408-6fac8c483071-ovn-node-metrics-cert\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-ovnkube-script-lib\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653919    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k96cq\""
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-dbus\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-system-cni-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.653978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-cni-bin\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-etc-kubernetes\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1eac53d2-0b0c-48bb-9076-4671d4cd8727-iptables-alerter-script\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654094 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-node-log\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-run-ovn-kubernetes\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654148    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-ovnkube-config\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-cnibin\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-hostroot\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654230    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654246    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-multus-certs\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1eac53d2-0b0c-48bb-9076-4671d4cd8727-host-slash\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654343    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctgj2\" (UniqueName: \"kubernetes.io/projected/1eac53d2-0b0c-48bb-9076-4671d4cd8727-kube-api-access-ctgj2\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-var-lib-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.654480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-kubelet-config\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fba2bd14-caf9-4175-a048-158d4665646a-konnectivity-ca\") pod \"konnectivity-agent-xdppf\" (UID: \"fba2bd14-caf9-4175-a048-158d4665646a\") " pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-cni-multus\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-kubelet\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-kubelet\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-ovn\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be159616-10cd-4bbc-9db6-dccb069099cd-multus-daemon-config\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-systemd-units\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-run-netns\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-etc-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-cni-bin\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-k8s-cni-cncf-io\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-netns\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654890    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffbq5\" (UniqueName: \"kubernetes.io/projected/748bdf6e-1c14-4269-b408-6fac8c483071-kube-api-access-ffbq5\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.655127 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.654930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-slash\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.655977 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.655039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fba2bd14-caf9-4175-a048-158d4665646a-agent-certs\") pod \"konnectivity-agent-xdppf\" (UID: \"fba2bd14-caf9-4175-a048-158d4665646a\") " pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.655977 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.655066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:06.655977 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.655553    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.656133 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.656019    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.656133 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.656073    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hf4cz\""
Sep 14 07:49:06.656390 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.656370    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.657757 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.657405    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.657757 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.657621    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.657757 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.657653    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wgqx9\""
Sep 14 07:49:06.658859 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.658287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.658859 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.658806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sxsxh\""
Sep 14 07:49:06.659020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.659002    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 07:49:06.659243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.659189    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 07:49:06.659863 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.659487    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.660315 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.660261    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:49:06.660826 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.660757    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 07:49:06.660926 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.660845    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:06.660926 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.660911    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:06.661044 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.661007    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.661403 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.661382    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bxgjx\""
Sep 14 07:49:06.661493 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.661418    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.662355 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.662299    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wvkx4\""
Sep 14 07:49:06.662476 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.662373    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 07:49:06.662663 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.662507    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 07:49:06.662928 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.662721    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 07:49:06.686893 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.686871    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6vjjs"
Sep 14 07:49:06.696028 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.696008    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6vjjs"
Sep 14 07:49:06.743296 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.743272    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 07:49:06.756121 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-run-netns\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysctl-conf\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-cni-bin\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-modprobe-d\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-k8s-cni-cncf-io\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-netns\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50e42341-0684-4293-bb7e-24e39e6be788-etc-tuned\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-run-netns\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fba2bd14-caf9-4175-a048-158d4665646a-agent-certs\") pod \"konnectivity-agent-xdppf\" (UID: \"fba2bd14-caf9-4175-a048-158d4665646a\") " pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756305    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-k8s-cni-cncf-io\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-cni-bin\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-cni-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-netns\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-conf-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-systemd\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-cni-netd\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-cni-netd\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756526    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-conf-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/32354ba1-68bb-449a-88b5-629a449b78be-serviceca\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-systemd\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-hosts-file\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.756642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-socket-dir-parent\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-cni-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg6wp\" (UniqueName: \"kubernetes.io/projected/be159616-10cd-4bbc-9db6-dccb069099cd-kube-api-access-sg6wp\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-systemd\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756721    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92knr\" (UniqueName: \"kubernetes.io/projected/50e42341-0684-4293-bb7e-24e39e6be788-kube-api-access-92knr\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-multus-socket-dir-parent\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-system-cni-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-etc-kubernetes\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1eac53d2-0b0c-48bb-9076-4671d4cd8727-iptables-alerter-script\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-run-ovn-kubernetes\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-ovnkube-config\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-kubelet-config\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-system-cni-dir\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-etc-kubernetes\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.756988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-run-ovn-kubernetes\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757002    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-kubelet-config\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.757604 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757101    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-sys\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-cnibin\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-hostroot\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1eac53d2-0b0c-48bb-9076-4671d4cd8727-host-slash\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctgj2\" (UniqueName: \"kubernetes.io/projected/1eac53d2-0b0c-48bb-9076-4671d4cd8727-kube-api-access-ctgj2\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cni-binary-copy\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32354ba1-68bb-449a-88b5-629a449b78be-host\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-tmp-dir\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fba2bd14-caf9-4175-a048-158d4665646a-konnectivity-ca\") pod \"konnectivity-agent-xdppf\" (UID: \"fba2bd14-caf9-4175-a048-158d4665646a\") " pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-cni-multus\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-ovnkube-config\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1eac53d2-0b0c-48bb-9076-4671d4cd8727-host-slash\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1eac53d2-0b0c-48bb-9076-4671d4cd8727-iptables-alerter-script\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-kubelet\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-cni-multus\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-hostroot\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-cnibin\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.759093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-kubelet\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-kubelet\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-kubelet\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-ovn\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-run-ovn\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfl2n\" (UniqueName: \"kubernetes.io/projected/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-kube-api-access-lfl2n\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-sys-fs\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be159616-10cd-4bbc-9db6-dccb069099cd-multus-daemon-config\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-systemd-units\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-etc-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-etc-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fba2bd14-caf9-4175-a048-158d4665646a-konnectivity-ca\") pod \"konnectivity-agent-xdppf\" (UID: \"fba2bd14-caf9-4175-a048-158d4665646a\") " pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.757981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-var-lib-kubelet\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50e42341-0684-4293-bb7e-24e39e6be788-tmp\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.760068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-socket-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-systemd-units\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffbq5\" (UniqueName: \"kubernetes.io/projected/748bdf6e-1c14-4269-b408-6fac8c483071-kube-api-access-ffbq5\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-system-cni-dir\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-lib-modules\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/748bdf6e-1c14-4269-b408-6fac8c483071-ovn-node-metrics-cert\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-dbus\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5stf\" (UniqueName: \"kubernetes.io/projected/21b8f02f-945b-4f0b-84bf-98a0053d7f18-kube-api-access-c5stf\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-slash\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-env-overrides\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khh8x\" (UniqueName: \"kubernetes.io/projected/32354ba1-68bb-449a-88b5-629a449b78be-kube-api-access-khh8x\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysctl-d\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/be159616-10cd-4bbc-9db6-dccb069099cd-multus-daemon-config\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be159616-10cd-4bbc-9db6-dccb069099cd-cni-binary-copy\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-log-socket\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.760738 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-ovnkube-script-lib\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-device-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-654fw\" (UniqueName: \"kubernetes.io/projected/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-kube-api-access-654fw\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-run\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-cni-bin\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-node-log\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-kubernetes\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-multus-certs\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-var-lib-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-registration-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysconfig\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cnibin\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl678\" (UniqueName: \"kubernetes.io/projected/223808bc-d985-4ead-8463-c63a3972b62b-kube-api-access-kl678\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.758967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-host\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.761392 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.759136    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.759216    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:07.259194494 +0000 UTC m=+2.967208641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-log-socket\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759487    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/be159616-10cd-4bbc-9db6-dccb069099cd-cni-binary-copy\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-run-multus-certs\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-var-lib-openvswitch\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-host-slash\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/748bdf6e-1c14-4269-b408-6fac8c483071-node-log\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/be159616-10cd-4bbc-9db6-dccb069099cd-host-var-lib-cni-bin\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-dbus\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.759966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-ovnkube-script-lib\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.760221    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/748bdf6e-1c14-4269-b408-6fac8c483071-env-overrides\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.761588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/748bdf6e-1c14-4269-b408-6fac8c483071-ovn-node-metrics-cert\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.762279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.761705    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fba2bd14-caf9-4175-a048-158d4665646a-agent-certs\") pod \"konnectivity-agent-xdppf\" (UID: \"fba2bd14-caf9-4175-a048-158d4665646a\") " pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.766248 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.765975    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:06.766248 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.766004    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:06.766248 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.766018    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:06.766248 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.766084    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:07.266064751 +0000 UTC m=+2.974078876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:06.767462 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.767442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctgj2\" (UniqueName: \"kubernetes.io/projected/1eac53d2-0b0c-48bb-9076-4671d4cd8727-kube-api-access-ctgj2\") pod \"iptables-alerter-wj55g\" (UID: \"1eac53d2-0b0c-48bb-9076-4671d4cd8727\") " pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.767865 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.767840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffbq5\" (UniqueName: \"kubernetes.io/projected/748bdf6e-1c14-4269-b408-6fac8c483071-kube-api-access-ffbq5\") pod \"ovnkube-node-hl96f\" (UID: \"748bdf6e-1c14-4269-b408-6fac8c483071\") " pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.768672 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.768646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg6wp\" (UniqueName: \"kubernetes.io/projected/be159616-10cd-4bbc-9db6-dccb069099cd-kube-api-access-sg6wp\") pod \"multus-xtts5\" (UID: \"be159616-10cd-4bbc-9db6-dccb069099cd\") " pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.786700 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.786651    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" event={"ID":"2ac03057413443baa8a950301b2f270d","Type":"ContainerStarted","Data":"9a2a4baadd024f26c89a524e27f745a94cd68e22447b3a326195535dbd13839b"}
Sep 14 07:49:06.788030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.788002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" event={"ID":"c813448e0dd7de1c8f6d146bb680f0b3","Type":"ContainerStarted","Data":"72dcd73d63d61a486b760a4d71a9d0c93129e0d17fadec5ca620a3658c88540a"}
Sep 14 07:49:06.859243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859211    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5stf\" (UniqueName: \"kubernetes.io/projected/21b8f02f-945b-4f0b-84bf-98a0053d7f18-kube-api-access-c5stf\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.859395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khh8x\" (UniqueName: \"kubernetes.io/projected/32354ba1-68bb-449a-88b5-629a449b78be-kube-api-access-khh8x\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.859395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysctl-d\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-device-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.859395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-654fw\" (UniqueName: \"kubernetes.io/projected/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-kube-api-access-654fw\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.859395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859364    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-run\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-kubernetes\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-registration-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysconfig\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysctl-d\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cnibin\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysconfig\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-kubernetes\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-registration-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-device-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-run\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl678\" (UniqueName: \"kubernetes.io/projected/223808bc-d985-4ead-8463-c63a3972b62b-kube-api-access-kl678\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cnibin\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.859711 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-host\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysctl-conf\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-modprobe-d\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-host\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50e42341-0684-4293-bb7e-24e39e6be788-etc-tuned\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/32354ba1-68bb-449a-88b5-629a449b78be-serviceca\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859880    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-hosts-file\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-modprobe-d\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-systemd\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-sysctl-conf\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-etc-systemd\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859997    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-hosts-file\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.859987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92knr\" (UniqueName: \"kubernetes.io/projected/50e42341-0684-4293-bb7e-24e39e6be788-kube-api-access-92knr\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-sys\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cni-binary-copy\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860097    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-sys\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.860485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32354ba1-68bb-449a-88b5-629a449b78be-host\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-tmp-dir\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfl2n\" (UniqueName: \"kubernetes.io/projected/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-kube-api-access-lfl2n\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-sys-fs\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860315    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-var-lib-kubelet\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50e42341-0684-4293-bb7e-24e39e6be788-tmp\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860364    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-socket-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/32354ba1-68bb-449a-88b5-629a449b78be-serviceca\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-system-cni-dir\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-lib-modules\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-tmp-dir\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860535    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-sys-fs\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-lib-modules\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.861465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860636    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50e42341-0684-4293-bb7e-24e39e6be788-var-lib-kubelet\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cni-binary-copy\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/32354ba1-68bb-449a-88b5-629a449b78be-host\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.860827    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21b8f02f-945b-4f0b-84bf-98a0053d7f18-system-cni-dir\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:06.860900    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:07.360880297 +0000 UTC m=+3.068894425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.860972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-socket-dir\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.861023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.862348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.861193    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21b8f02f-945b-4f0b-84bf-98a0053d7f18-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.862865 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.862689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/50e42341-0684-4293-bb7e-24e39e6be788-etc-tuned\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.863199 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.863177    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/50e42341-0684-4293-bb7e-24e39e6be788-tmp\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.869227 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.869202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-654fw\" (UniqueName: \"kubernetes.io/projected/55b7e9ee-dd40-44c3-9cd8-55f8929de5c4-kube-api-access-654fw\") pod \"aws-ebs-csi-driver-node-ctzjw\" (UID: \"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:06.869737 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.869713    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5stf\" (UniqueName: \"kubernetes.io/projected/21b8f02f-945b-4f0b-84bf-98a0053d7f18-kube-api-access-c5stf\") pod \"multus-additional-cni-plugins-5gctk\" (UID: \"21b8f02f-945b-4f0b-84bf-98a0053d7f18\") " pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:06.869833 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.869796    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khh8x\" (UniqueName: \"kubernetes.io/projected/32354ba1-68bb-449a-88b5-629a449b78be-kube-api-access-khh8x\") pod \"node-ca-2jjqz\" (UID: \"32354ba1-68bb-449a-88b5-629a449b78be\") " pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:06.869891 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.869855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92knr\" (UniqueName: \"kubernetes.io/projected/50e42341-0684-4293-bb7e-24e39e6be788-kube-api-access-92knr\") pod \"tuned-72bzr\" (UID: \"50e42341-0684-4293-bb7e-24e39e6be788\") " pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.870610 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.870589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl678\" (UniqueName: \"kubernetes.io/projected/223808bc-d985-4ead-8463-c63a3972b62b-kube-api-access-kl678\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:06.870976 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.870950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfl2n\" (UniqueName: \"kubernetes.io/projected/fb587b36-5ad8-4a6e-bce6-9d00228fdef6-kube-api-access-lfl2n\") pod \"node-resolver-j786z\" (UID: \"fb587b36-5ad8-4a6e-bce6-9d00228fdef6\") " pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:06.956980 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.956942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:06.967349 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.967324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xtts5"
Sep 14 07:49:06.976098 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.976073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wj55g"
Sep 14 07:49:06.984062 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.984042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:06.991917 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.991893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-72bzr"
Sep 14 07:49:06.999593 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:06.999575    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j786z"
Sep 14 07:49:07.007284 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.007263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5gctk"
Sep 14 07:49:07.015959 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.015943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2jjqz"
Sep 14 07:49:07.022591 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.022572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw"
Sep 14 07:49:07.184355 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.184334    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:49:07.264002 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.263959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:07.264189 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.264111    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:07.264189 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.264185    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.264165353 +0000 UTC m=+3.972179479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:07.364874 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.364840    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:07.365075 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.364884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:07.365075 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.365034    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:07.365175 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.365123    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.365099409 +0000 UTC m=+4.073113542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:07.365175 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.365038    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:07.365175 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.365154    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:07.365175 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.365165    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:07.365365 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.365209    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.365197845 +0000 UTC m=+4.073211986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:07.698490 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.698422    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:44:07 +0000 UTC" deadline="2028-04-27 08:16:08.226326264 +0000 UTC"
Sep 14 07:49:07.698490 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.698489    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14184h27m0.527851396s"
Sep 14 07:49:07.783054 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.783023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:07.783229 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:07.783023    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:07.783229 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.783123    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:07.783348 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:07.783240    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:07.840163 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.840130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba2bd14_caf9_4175_a048_158d4665646a.slice/crio-aa0180799bff18442c3cabcbbbe0ad5730623c0f8be980f4a462a037c010016f WatchSource:0}: Error finding container aa0180799bff18442c3cabcbbbe0ad5730623c0f8be980f4a462a037c010016f: Status 404 returned error can't find the container with id aa0180799bff18442c3cabcbbbe0ad5730623c0f8be980f4a462a037c010016f
Sep 14 07:49:07.841899 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.841748    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748bdf6e_1c14_4269_b408_6fac8c483071.slice/crio-88ac7241ed755433bcbcc57ff5a30d2dc4288fbb9bdd2870f594c9a82528f62c WatchSource:0}: Error finding container 88ac7241ed755433bcbcc57ff5a30d2dc4288fbb9bdd2870f594c9a82528f62c: Status 404 returned error can't find the container with id 88ac7241ed755433bcbcc57ff5a30d2dc4288fbb9bdd2870f594c9a82528f62c
Sep 14 07:49:07.846370 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.846348    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e42341_0684_4293_bb7e_24e39e6be788.slice/crio-e017ab13cb2bc7e65abb990d89a397d8b2f46c5f90168ffb701d01d523e3a379 WatchSource:0}: Error finding container e017ab13cb2bc7e65abb990d89a397d8b2f46c5f90168ffb701d01d523e3a379: Status 404 returned error can't find the container with id e017ab13cb2bc7e65abb990d89a397d8b2f46c5f90168ffb701d01d523e3a379
Sep 14 07:49:07.847656 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.847632    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe159616_10cd_4bbc_9db6_dccb069099cd.slice/crio-852b4a7b47e035f3be0c31932e263b9036664e521a62131ab5e5b7e6ce31a8ef WatchSource:0}: Error finding container 852b4a7b47e035f3be0c31932e263b9036664e521a62131ab5e5b7e6ce31a8ef: Status 404 returned error can't find the container with id 852b4a7b47e035f3be0c31932e263b9036664e521a62131ab5e5b7e6ce31a8ef
Sep 14 07:49:07.849031 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.849010    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b8f02f_945b_4f0b_84bf_98a0053d7f18.slice/crio-59870ceace84b1173614b82f4915c50ea70bc73a36a1a750889903eca753df07 WatchSource:0}: Error finding container 59870ceace84b1173614b82f4915c50ea70bc73a36a1a750889903eca753df07: Status 404 returned error can't find the container with id 59870ceace84b1173614b82f4915c50ea70bc73a36a1a750889903eca753df07
Sep 14 07:49:07.850911 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.850717    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32354ba1_68bb_449a_88b5_629a449b78be.slice/crio-727ea69df078f0f147036f6c159ba74f6d717fa5a79830e5dd2b90fe85deb932 WatchSource:0}: Error finding container 727ea69df078f0f147036f6c159ba74f6d717fa5a79830e5dd2b90fe85deb932: Status 404 returned error can't find the container with id 727ea69df078f0f147036f6c159ba74f6d717fa5a79830e5dd2b90fe85deb932
Sep 14 07:49:07.851394 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.851344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eac53d2_0b0c_48bb_9076_4671d4cd8727.slice/crio-bbcb5bd923ef1b02e4853016ea4a86df33793e0b8ac3baf11e1da8f8227f3c06 WatchSource:0}: Error finding container bbcb5bd923ef1b02e4853016ea4a86df33793e0b8ac3baf11e1da8f8227f3c06: Status 404 returned error can't find the container with id bbcb5bd923ef1b02e4853016ea4a86df33793e0b8ac3baf11e1da8f8227f3c06
Sep 14 07:49:07.853251 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.852500    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b7e9ee_dd40_44c3_9cd8_55f8929de5c4.slice/crio-10f367a536ea364641d132dca1fcb80449c1121072ea5bf41cdeb31db4f12574 WatchSource:0}: Error finding container 10f367a536ea364641d132dca1fcb80449c1121072ea5bf41cdeb31db4f12574: Status 404 returned error can't find the container with id 10f367a536ea364641d132dca1fcb80449c1121072ea5bf41cdeb31db4f12574
Sep 14 07:49:07.853251 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:07.853175    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb587b36_5ad8_4a6e_bce6_9d00228fdef6.slice/crio-4e7fde13b9e9c0660cea8db4562e12b8933b221822c9474b8391a256d24db9c0 WatchSource:0}: Error finding container 4e7fde13b9e9c0660cea8db4562e12b8933b221822c9474b8391a256d24db9c0: Status 404 returned error can't find the container with id 4e7fde13b9e9c0660cea8db4562e12b8933b221822c9474b8391a256d24db9c0
Sep 14 07:49:08.273420 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.273278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:08.273604 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.273482    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:08.273604 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.273564    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.273541765 +0000 UTC m=+5.981555899 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:08.374954 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.374563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:08.374954 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.374625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:08.374954 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.374799    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:08.374954 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.374822    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:08.374954 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.374835    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:08.374954 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.374900    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.374881643 +0000 UTC m=+6.082895782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:08.375399 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.375314    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:08.375399 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.375365    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.375347977 +0000 UTC m=+6.083362111 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:08.701858 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.698959    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:44:07 +0000 UTC" deadline="2028-04-11 08:35:29.108173909 +0000 UTC"
Sep 14 07:49:08.701858 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.699019    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13800h46m20.409158878s"
Sep 14 07:49:08.784640 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.784606    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:08.784821 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:08.784746    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:08.798191 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.796764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xdppf" event={"ID":"fba2bd14-caf9-4175-a048-158d4665646a","Type":"ContainerStarted","Data":"aa0180799bff18442c3cabcbbbe0ad5730623c0f8be980f4a462a037c010016f"}
Sep 14 07:49:08.800944 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.800912    2563 generic.go:356] "Generic (PLEG): container finished" podID="c813448e0dd7de1c8f6d146bb680f0b3" containerID="abbf67e006c844ceaf64fcda8db026d660c1476d1badaa90d5858c8f402699fb" exitCode=0
Sep 14 07:49:08.801196 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.801010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" event={"ID":"c813448e0dd7de1c8f6d146bb680f0b3","Type":"ContainerDied","Data":"abbf67e006c844ceaf64fcda8db026d660c1476d1badaa90d5858c8f402699fb"}
Sep 14 07:49:08.808709 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.808682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw" event={"ID":"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4","Type":"ContainerStarted","Data":"10f367a536ea364641d132dca1fcb80449c1121072ea5bf41cdeb31db4f12574"}
Sep 14 07:49:08.814084 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.814056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2jjqz" event={"ID":"32354ba1-68bb-449a-88b5-629a449b78be","Type":"ContainerStarted","Data":"727ea69df078f0f147036f6c159ba74f6d717fa5a79830e5dd2b90fe85deb932"}
Sep 14 07:49:08.830259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.827979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerStarted","Data":"59870ceace84b1173614b82f4915c50ea70bc73a36a1a750889903eca753df07"}
Sep 14 07:49:08.834030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.834003    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wj55g" event={"ID":"1eac53d2-0b0c-48bb-9076-4671d4cd8727","Type":"ContainerStarted","Data":"bbcb5bd923ef1b02e4853016ea4a86df33793e0b8ac3baf11e1da8f8227f3c06"}
Sep 14 07:49:08.840133 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.840110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-72bzr" event={"ID":"50e42341-0684-4293-bb7e-24e39e6be788","Type":"ContainerStarted","Data":"e017ab13cb2bc7e65abb990d89a397d8b2f46c5f90168ffb701d01d523e3a379"}
Sep 14 07:49:08.847335 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.846643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" event={"ID":"2ac03057413443baa8a950301b2f270d","Type":"ContainerStarted","Data":"7dd0eaba8aadfa18a572699b45c2bc90e29bed9a3f8a4ee93c38ceb8da773eed"}
Sep 14 07:49:08.864013 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.863980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j786z" event={"ID":"fb587b36-5ad8-4a6e-bce6-9d00228fdef6","Type":"ContainerStarted","Data":"4e7fde13b9e9c0660cea8db4562e12b8933b221822c9474b8391a256d24db9c0"}
Sep 14 07:49:08.864742 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.864687    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-172.ec2.internal" podStartSLOduration=2.8646692209999998 podStartE2EDuration="2.864669221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:08.864238323 +0000 UTC m=+4.572252466" watchObservedRunningTime="2026-09-14 07:49:08.864669221 +0000 UTC m=+4.572683365"
Sep 14 07:49:08.873019 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.872954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xtts5" event={"ID":"be159616-10cd-4bbc-9db6-dccb069099cd","Type":"ContainerStarted","Data":"852b4a7b47e035f3be0c31932e263b9036664e521a62131ab5e5b7e6ce31a8ef"}
Sep 14 07:49:08.896374 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:08.896340    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"88ac7241ed755433bcbcc57ff5a30d2dc4288fbb9bdd2870f594c9a82528f62c"}
Sep 14 07:49:09.783221 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:09.782288    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:09.783221 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:09.782713    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:09.783221 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:09.782453    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:09.783221 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:09.783176    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:09.914987 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:09.914882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" event={"ID":"c813448e0dd7de1c8f6d146bb680f0b3","Type":"ContainerStarted","Data":"f97deae03d98c2bf7b8d12e222ecd53ac8c2c964fb8ee83126d48e16bd688b41"}
Sep 14 07:49:10.296846 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:10.296162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:10.296846 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.296389    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:10.296846 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.296459    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:14.296439683 +0000 UTC m=+10.004453804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:10.397752 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:10.397518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:10.397752 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:10.397614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:10.397752 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.397722    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:10.397752 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.397770    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:10.398131 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.397786    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:10.398131 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.397855    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:14.397834575 +0000 UTC m=+10.105848699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:10.398448 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.397729    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:10.398448 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.398310    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:14.398292174 +0000 UTC m=+10.106306296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:10.782902 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:10.782857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:10.783109 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:10.783004    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:11.782911 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:11.782490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:11.782911 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:11.782591    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:11.782911 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:11.782652    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:11.782911 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:11.782795    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:12.783456 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:12.782942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:12.783456 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:12.783083    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:13.782394 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:13.782316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:13.782568 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:13.782417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:13.782568 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:13.782546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:13.782660 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:13.782638    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:14.332714 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:14.332677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:14.333245 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.332862    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:14.333245 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.332932    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:22.332911283 +0000 UTC m=+18.040925404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:14.433464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:14.433604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.433657    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.433685    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.433699    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.433726    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.433803    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:22.433775664 +0000 UTC m=+18.141789798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:14.433912 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.433877    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:22.433856893 +0000 UTC m=+18.141871015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:14.784795 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:14.784004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:14.784795 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:14.784138    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:15.783072 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:15.783036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:15.783548 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:15.783180    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:15.783616 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:15.783036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:15.783699 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:15.783659    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:16.782838 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:16.782805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:16.783022 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:16.782941    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:17.782154 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:17.782121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:17.782154 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:17.782141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:17.782623 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:17.782241    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:17.782623 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:17.782366    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:18.783130 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:18.783047    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:18.783540 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:18.783192    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:19.782267 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:19.782235    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:19.782430 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:19.782275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:19.782430 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:19.782353    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:19.782535 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:19.782471    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:20.782555 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:20.782519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:20.783043 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:20.782647    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:21.782950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:21.782917    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:21.783419 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:21.783023    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:21.783419 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:21.783084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:21.783419 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:21.783215    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:22.395029 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:22.394991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:22.395211 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.395138    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:22.395211 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.395192    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:38.395179315 +0000 UTC m=+34.103193446 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:22.496010 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:22.495979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:22.496010 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:22.496027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:22.496237 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.496154    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:22.496237 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.496164    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:22.496237 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.496178    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:22.496237 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.496190    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:22.496237 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.496217    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:38.496200383 +0000 UTC m=+34.204214516 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:22.496237 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.496238    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:38.49622824 +0000 UTC m=+34.204242360 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:22.783101 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:22.783057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:22.783573 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:22.783223    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:23.782913 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:23.782875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:23.783081 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:23.783014    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:23.783081 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:23.783055    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:23.783482 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:23.783122    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:24.783570 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:24.783522    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:24.784000 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:24.783628    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:25.782925 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:25.782891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:25.783097 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:25.782945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:25.783097 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:25.783034    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:25.783243 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:25.783120    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:26.783041 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.782688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:26.783808 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:26.783128    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:26.949029 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.949001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j786z" event={"ID":"fb587b36-5ad8-4a6e-bce6-9d00228fdef6","Type":"ContainerStarted","Data":"9e963faf928658e0e3c4606380e638083cdac0edb4906345e0af5221728a7c87"}
Sep 14 07:49:26.950508 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.950486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xtts5" event={"ID":"be159616-10cd-4bbc-9db6-dccb069099cd","Type":"ContainerStarted","Data":"ad26c067c9ff47716f856e3eae9f977e253d96d8a230cbe199a009d2bc9e1986"}
Sep 14 07:49:26.953662 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.953636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"910efbee5ff7938209d9281ed5c50cd47b0607c022991d0cac8053375e2da49c"}
Sep 14 07:49:26.953756 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.953700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"5f113700604431b81b8783cc992647b2047a4d1a5178b4b0d44efa53a5cdc317"}
Sep 14 07:49:26.953810 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.953760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"182a3f00272f25ec072c949b26e6e29203bc9d62b1a7a0a0602809903eb1e40b"}
Sep 14 07:49:26.953810 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.953802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"788e9ef3032ce10924dda88be7287a258d014d228b4366d1c62843fec6353078"}
Sep 14 07:49:26.953901 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.953830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"b9a469f347c5538b6ee592cfe2dfecc00e43ea37730508f349684e9417eaafe1"}
Sep 14 07:49:26.953901 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.953873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"942f5a82472c4f0a05d527da00913f277ad1ba083348eeb95cf606b92f964dd3"}
Sep 14 07:49:26.955033 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.955011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xdppf" event={"ID":"fba2bd14-caf9-4175-a048-158d4665646a","Type":"ContainerStarted","Data":"5ced07e227df733b7cd9581e6660c4fc9f09abd2f0e85499f495db3c17bfe734"}
Sep 14 07:49:26.956305 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.956288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw" event={"ID":"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4","Type":"ContainerStarted","Data":"08f01da4750ade522211c39dde416e82ad1bfd28af6cb96c2d608033a8ce8041"}
Sep 14 07:49:26.957516 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.957498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2jjqz" event={"ID":"32354ba1-68bb-449a-88b5-629a449b78be","Type":"ContainerStarted","Data":"84c9d34aa1773596d087013e564e0ff3dff550ef476b8516457065f63a59b609"}
Sep 14 07:49:26.958842 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.958825    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8f02f-945b-4f0b-84bf-98a0053d7f18" containerID="0ccdf64621bc3a84375840a47281ac30a525664e923eb29a339d2e2c0ee2e625" exitCode=0
Sep 14 07:49:26.958909 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.958884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerDied","Data":"0ccdf64621bc3a84375840a47281ac30a525664e923eb29a339d2e2c0ee2e625"}
Sep 14 07:49:26.960308 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.960289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-72bzr" event={"ID":"50e42341-0684-4293-bb7e-24e39e6be788","Type":"ContainerStarted","Data":"71620f0b2461ecefb7e46277288bf83028a7769ba986167e97d4bf32972e520b"}
Sep 14 07:49:26.965179 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.965146    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j786z" podStartSLOduration=3.934136333 podStartE2EDuration="21.965114927s" totalImagesPullingTime="18.030978594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.964389603 +0000 UTC m=+22.672403742" watchObservedRunningTime="2026-09-14 07:49:26.965114927 +0000 UTC m=+22.673129064"
Sep 14 07:49:26.965736 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.965703    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-172.ec2.internal" podStartSLOduration=20.965694489 podStartE2EDuration="20.965694489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:09.935127398 +0000 UTC m=+5.643141535" watchObservedRunningTime="2026-09-14 07:49:26.965694489 +0000 UTC m=+22.673708642"
Sep 14 07:49:26.978676 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.978641    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xdppf" podStartSLOduration=11.201015225 podStartE2EDuration="21.978629828s" totalImagesPullingTime="10.777614603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.97844947 +0000 UTC m=+22.686463600" watchObservedRunningTime="2026-09-14 07:49:26.978629828 +0000 UTC m=+22.686643968"
Sep 14 07:49:26.979512 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.979495    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:26.997401 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:26.997370    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xtts5" podStartSLOduration=3.906398437 podStartE2EDuration="21.997359913s" totalImagesPullingTime="18.090961476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.997005191 +0000 UTC m=+22.705019330" watchObservedRunningTime="2026-09-14 07:49:26.997359913 +0000 UTC m=+22.705374051"
Sep 14 07:49:27.013739 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.013698    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-72bzr" podStartSLOduration=3.975427925 podStartE2EDuration="22.013688633s" totalImagesPullingTime="18.038260708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:27.013674883 +0000 UTC m=+22.721689021" watchObservedRunningTime="2026-09-14 07:49:27.013688633 +0000 UTC m=+22.721702771"
Sep 14 07:49:27.051568 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.051488    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2jjqz" podStartSLOduration=4.018132705 podStartE2EDuration="22.051476922s" totalImagesPullingTime="18.033344217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:27.051415487 +0000 UTC m=+22.759429625" watchObservedRunningTime="2026-09-14 07:49:27.051476922 +0000 UTC m=+22.759491063"
Sep 14 07:49:27.327561 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.327536    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 07:49:27.710308 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.710202    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T07:49:27.327552619Z","UUID":"c004ac1c-b67e-45f4-b9d8-91c186218db9","Handler":null,"Name":"","Endpoint":""}
Sep 14 07:49:27.711848 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.711828    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 07:49:27.712015 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.711859    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 07:49:27.782421 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.782397    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:27.782570 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:27.782524    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:27.782683 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.782583    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:27.782765 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:27.782695    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:27.965483 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.965227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wj55g" event={"ID":"1eac53d2-0b0c-48bb-9076-4671d4cd8727","Type":"ContainerStarted","Data":"757317eb5f96238a14e4d520b92f078dd68f35ed29a794c232dd5cf569dc7be4"}
Sep 14 07:49:27.967681 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.967649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw" event={"ID":"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4","Type":"ContainerStarted","Data":"cb1e00bfed2daffe4d8927d40f25cbb56ee180035020069b89dd9e8fc0e9daa0"}
Sep 14 07:49:27.984210 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:27.984159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wj55g" podStartSLOduration=4.953240438 podStartE2EDuration="22.984139593s" totalImagesPullingTime="18.030899155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:27.983498161 +0000 UTC m=+23.691512300" watchObservedRunningTime="2026-09-14 07:49:27.984139593 +0000 UTC m=+23.692153733"
Sep 14 07:49:28.783079 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:28.783046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:28.783240 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:28.783188    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:28.974154 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:28.974113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"3a67033c57adb0e9cd58764a89e16d42f28add40515a9b26f84b5ea8648df4a8"}
Sep 14 07:49:28.976470 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:28.976444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw" event={"ID":"55b7e9ee-dd40-44c3-9cd8-55f8929de5c4","Type":"ContainerStarted","Data":"240742f2795d257c64b9816ebd6a6aae0535f7897d365911792be2a27a18e560"}
Sep 14 07:49:28.997208 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:28.997158    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ctzjw" podStartSLOduration=3.771586974 podStartE2EDuration="23.997143454s" totalImagesPullingTime="20.22555648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:28.996437799 +0000 UTC m=+24.704451939" watchObservedRunningTime="2026-09-14 07:49:28.997143454 +0000 UTC m=+24.705157624"
Sep 14 07:49:29.692195 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:29.692164    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:29.692837 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:29.692811    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:29.782295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:29.782262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:29.782295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:29.782283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:29.782476 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:29.782367    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:29.782520 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:29.782483    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:29.979515 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:29.979452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xdppf"
Sep 14 07:49:30.782654 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:30.782404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:30.782813 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:30.782783    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:31.782796 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.782764    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:31.783472 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.782796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:31.783472 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:31.782859    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:31.783472 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:31.782922    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:31.985605 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.985428    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8f02f-945b-4f0b-84bf-98a0053d7f18" containerID="8f2b508eb0a06d3670fe8680564bae4ec718ee3e4d1a5515342769df4ee01d12" exitCode=0
Sep 14 07:49:31.985764 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.985479    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerDied","Data":"8f2b508eb0a06d3670fe8680564bae4ec718ee3e4d1a5515342769df4ee01d12"}
Sep 14 07:49:31.989520 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.989497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" event={"ID":"748bdf6e-1c14-4269-b408-6fac8c483071","Type":"ContainerStarted","Data":"5148eaeb37d1ca9bc4d49617ad988fb9f1f3c85a0d487383e11013d8a92740bf"}
Sep 14 07:49:31.989742 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.989706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:31.989853 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.989786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:31.989853 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:31.989835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:32.005192 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.005172    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:32.008215 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.008196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:49:32.039686 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.039598    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f" podStartSLOduration=8.938542247 podStartE2EDuration="27.039586874s" totalImagesPullingTime="18.101044627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:32.039352888 +0000 UTC m=+27.747367027" watchObservedRunningTime="2026-09-14 07:49:32.039586874 +0000 UTC m=+27.747601013"
Sep 14 07:49:32.782671 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.782646    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:32.782810 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:32.782786    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:32.994714 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.994642    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8f02f-945b-4f0b-84bf-98a0053d7f18" containerID="a8fa01d79bd357ff8a6afa075dc769eb9403faf02c0ee5a53749d25e6044290e" exitCode=0
Sep 14 07:49:32.994856 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.994737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerDied","Data":"a8fa01d79bd357ff8a6afa075dc769eb9403faf02c0ee5a53749d25e6044290e"}
Sep 14 07:49:32.996984 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.996960    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cxxtl"]
Sep 14 07:49:32.997080 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:32.997042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:32.997165 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:32.997139    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:33.000936 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:33.000915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s7jl4"]
Sep 14 07:49:33.001060 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:33.001031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:33.001144 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:33.001115    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:33.001626 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:33.001602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xmvb"]
Sep 14 07:49:33.001712 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:33.001696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:33.001808 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:33.001791    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:33.999783 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:33.999753    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8f02f-945b-4f0b-84bf-98a0053d7f18" containerID="b3e38bfa196d92e0976b97e7c4d195a2768be73a98d4fde261a84aff9202d137" exitCode=0
Sep 14 07:49:34.000058 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:33.999831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerDied","Data":"b3e38bfa196d92e0976b97e7c4d195a2768be73a98d4fde261a84aff9202d137"}
Sep 14 07:49:34.783173 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:34.783138    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:34.783339 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:34.783247    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:34.783339 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:34.783262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:34.783461 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:34.783334    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:34.783461 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:34.783390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:34.783560 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:34.783457    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:36.782434 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:36.782376    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:36.782914 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:36.782387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:36.782914 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:36.782478    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cxxtl" podUID="0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24"
Sep 14 07:49:36.782914 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:36.782566    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s7jl4" podUID="eff243d2-766c-46ab-9fdb-d8c8e4fabb4a"
Sep 14 07:49:36.782914 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:36.782389    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:36.782914 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:36.782677    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7xmvb" podUID="223808bc-d985-4ead-8463-c63a3972b62b"
Sep 14 07:49:38.405468 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.405441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:38.406064 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.405547    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:38.406064 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.405595    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret podName:0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:10.405582691 +0000 UTC m=+66.113596807 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret") pod "global-pull-secret-syncer-cxxtl" (UID: "0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:49:38.506360 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.506333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:38.506499 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.506373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:38.506499 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.506462    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:38.506567 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.506522    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs podName:223808bc-d985-4ead-8463-c63a3972b62b nodeName:}" failed. No retries permitted until 2026-09-14 07:50:10.506503196 +0000 UTC m=+66.214517336 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs") pod "network-metrics-daemon-7xmvb" (UID: "223808bc-d985-4ead-8463-c63a3972b62b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:49:38.506567 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.506553    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:49:38.506633 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.506569    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:49:38.506633 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.506581    2563 projected.go:196] Error preparing data for projected volume kube-api-access-k65zl for pod openshift-network-diagnostics/network-check-target-s7jl4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:38.506633 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.506621    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl podName:eff243d2-766c-46ab-9fdb-d8c8e4fabb4a nodeName:}" failed. No retries permitted until 2026-09-14 07:50:10.506606564 +0000 UTC m=+66.214620696 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k65zl" (UniqueName: "kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl") pod "network-check-target-s7jl4" (UID: "eff243d2-766c-46ab-9fdb-d8c8e4fabb4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:49:38.603512 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.603495    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-172.ec2.internal" event="NodeReady"
Sep 14 07:49:38.603664 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.603594    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 07:49:38.644559 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.644535    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9bdwp"]
Sep 14 07:49:38.657595 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.657532    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rzkr2"]
Sep 14 07:49:38.657721 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.657663    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.660062 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.660041    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 07:49:38.660170 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.660043    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd8p\""
Sep 14 07:49:38.660795 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.660485    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 07:49:38.671050 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.671031    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rzkr2"]
Sep 14 07:49:38.671160 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.671066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9bdwp"]
Sep 14 07:49:38.671218 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.671162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.673798 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.673780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 07:49:38.673895 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.673838    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8nsp\""
Sep 14 07:49:38.673957 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.673836    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 07:49:38.674014 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.673789    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 07:49:38.783172 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.783140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:49:38.783346 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.783140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:49:38.783438 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.783413    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:49:38.785908 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.785888    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:49:38.786005 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.785888    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:49:38.786005 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.785938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:49:38.786122 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.786010    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfw6g\""
Sep 14 07:49:38.786122 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.786045    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:49:38.786229 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.786151    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wkwvc\""
Sep 14 07:49:38.809071 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.809051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/af1713b8-c170-4d6b-a8d9-b53309b2ac75-config-volume\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.809186 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.809085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n5wj\" (UniqueName: \"kubernetes.io/projected/97683ac6-241b-47bc-b93d-2b921a7f7140-kube-api-access-4n5wj\") pod \"ingress-canary-rzkr2\" (UID: \"97683ac6-241b-47bc-b93d-2b921a7f7140\") " pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.809186 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.809111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.809186 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.809170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7gxh\" (UniqueName: \"kubernetes.io/projected/af1713b8-c170-4d6b-a8d9-b53309b2ac75-kube-api-access-l7gxh\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.809340 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.809210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97683ac6-241b-47bc-b93d-2b921a7f7140-cert\") pod \"ingress-canary-rzkr2\" (UID: \"97683ac6-241b-47bc-b93d-2b921a7f7140\") " pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.809340 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.809239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af1713b8-c170-4d6b-a8d9-b53309b2ac75-tmp-dir\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.909885 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.909822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.909885 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.909855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7gxh\" (UniqueName: \"kubernetes.io/projected/af1713b8-c170-4d6b-a8d9-b53309b2ac75-kube-api-access-l7gxh\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.909885 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.909871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97683ac6-241b-47bc-b93d-2b921a7f7140-cert\") pod \"ingress-canary-rzkr2\" (UID: \"97683ac6-241b-47bc-b93d-2b921a7f7140\") " pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.909885 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.909887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af1713b8-c170-4d6b-a8d9-b53309b2ac75-tmp-dir\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.910210 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.909975    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:38.910210 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:38.910050    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls podName:af1713b8-c170-4d6b-a8d9-b53309b2ac75 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:39.410029298 +0000 UTC m=+35.118043420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls") pod "dns-default-9bdwp" (UID: "af1713b8-c170-4d6b-a8d9-b53309b2ac75") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:38.910210 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.910071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/af1713b8-c170-4d6b-a8d9-b53309b2ac75-config-volume\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.910210 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.910115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n5wj\" (UniqueName: \"kubernetes.io/projected/97683ac6-241b-47bc-b93d-2b921a7f7140-kube-api-access-4n5wj\") pod \"ingress-canary-rzkr2\" (UID: \"97683ac6-241b-47bc-b93d-2b921a7f7140\") " pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.910399 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.910239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af1713b8-c170-4d6b-a8d9-b53309b2ac75-tmp-dir\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.910665 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.910646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/af1713b8-c170-4d6b-a8d9-b53309b2ac75-config-volume\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.913983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.913965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97683ac6-241b-47bc-b93d-2b921a7f7140-cert\") pod \"ingress-canary-rzkr2\" (UID: \"97683ac6-241b-47bc-b93d-2b921a7f7140\") " pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.919405 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.919384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n5wj\" (UniqueName: \"kubernetes.io/projected/97683ac6-241b-47bc-b93d-2b921a7f7140-kube-api-access-4n5wj\") pod \"ingress-canary-rzkr2\" (UID: \"97683ac6-241b-47bc-b93d-2b921a7f7140\") " pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:38.919702 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.919682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7gxh\" (UniqueName: \"kubernetes.io/projected/af1713b8-c170-4d6b-a8d9-b53309b2ac75-kube-api-access-l7gxh\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:38.979702 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:38.979684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rzkr2"
Sep 14 07:49:39.014846 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.014820    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8f02f-945b-4f0b-84bf-98a0053d7f18" containerID="43960399102ab93920436bf9f97fa4fabf3ea4f39a309f8f94e50de271f855e5" exitCode=0
Sep 14 07:49:39.014951 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.014872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerDied","Data":"43960399102ab93920436bf9f97fa4fabf3ea4f39a309f8f94e50de271f855e5"}
Sep 14 07:49:39.276960 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.276932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rzkr2"]
Sep 14 07:49:39.280325 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:39.280301    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97683ac6_241b_47bc_b93d_2b921a7f7140.slice/crio-6a7c8041f8ab91b7b500f509edc659ae4e7a7e4d4b74e0e1ccd5182899ad06e6 WatchSource:0}: Error finding container 6a7c8041f8ab91b7b500f509edc659ae4e7a7e4d4b74e0e1ccd5182899ad06e6: Status 404 returned error can't find the container with id 6a7c8041f8ab91b7b500f509edc659ae4e7a7e4d4b74e0e1ccd5182899ad06e6
Sep 14 07:49:39.412235 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.412208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:39.412761 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.412319    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:39.412761 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.412367    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls podName:af1713b8-c170-4d6b-a8d9-b53309b2ac75 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.412352578 +0000 UTC m=+36.120366695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls") pod "dns-default-9bdwp" (UID: "af1713b8-c170-4d6b-a8d9-b53309b2ac75") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:39.580881 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.580857    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-8lgjz"]
Sep 14 07:49:39.596045 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.596022    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-f9dtl"]
Sep 14 07:49:39.596151 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.596030    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.598965 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.598804    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 14 07:49:39.598965 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.598822    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.598965 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.598860    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.598965 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.598822    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 14 07:49:39.599337 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.599316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-c46g8\""
Sep 14 07:49:39.604058 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.604041    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 14 07:49:39.609924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.609907    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-f9dtl"]
Sep 14 07:49:39.610000 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.609932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-8lgjz"]
Sep 14 07:49:39.610037 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.609999    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.613309 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.613290    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 14 07:49:39.613408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.613313    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zfggm\""
Sep 14 07:49:39.613408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.613318    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.613408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.613355    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 14 07:49:39.613408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.613401    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.618224 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.618207    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 14 07:49:39.678664 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.678639    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"]
Sep 14 07:49:39.695933 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.695908    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55d4864867-9grlk"]
Sep 14 07:49:39.696041 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.696019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:39.698552 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.698531    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 14 07:49:39.698653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.698557    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:49:39.698653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.698622    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 14 07:49:39.698803 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.698788    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-flrlx\""
Sep 14 07:49:39.706024 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.706002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"]
Sep 14 07:49:39.706120 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.706099    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.707852 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.707831    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d4864867-9grlk"]
Sep 14 07:49:39.708502 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.708486    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 07:49:39.708699 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.708679    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 07:49:39.708817 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.708758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfxc9\""
Sep 14 07:49:39.708817 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.708775    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 07:49:39.713382 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-tmp\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.713466 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-snapshots\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.713466 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-serving-cert\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.713466 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpc5x\" (UniqueName: \"kubernetes.io/projected/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-kube-api-access-tpc5x\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.713466 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71927f5a-bf06-4703-9470-8398d43031b0-trusted-ca\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.713684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71927f5a-bf06-4703-9470-8398d43031b0-serving-cert\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.713684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r29dj\" (UniqueName: \"kubernetes.io/projected/71927f5a-bf06-4703-9470-8398d43031b0-kube-api-access-r29dj\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.713684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713540    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 07:49:39.713684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-trusted-ca-bundle\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.713684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71927f5a-bf06-4703-9470-8398d43031b0-config\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.713684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.713618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-service-ca-bundle\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.814800 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-snapshots\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.814924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-serving-cert\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.814924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814821    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpc5x\" (UniqueName: \"kubernetes.io/projected/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-kube-api-access-tpc5x\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.814924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71927f5a-bf06-4703-9470-8398d43031b0-trusted-ca\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.814924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-image-registry-private-configuration\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.814924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a0dc390-96dd-4763-889a-d7e8dd8ff645-ca-trust-extracted\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.814924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71927f5a-bf06-4703-9470-8398d43031b0-serving-cert\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.815165 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.814967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r29dj\" (UniqueName: \"kubernetes.io/projected/71927f5a-bf06-4703-9470-8398d43031b0-kube-api-access-r29dj\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.815165 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpspx\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-kube-api-access-jpspx\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.815165 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-trusted-ca-bundle\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.815295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-installation-pull-secrets\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.815356 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:39.815408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-bound-sa-token\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.815460 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-snapshots\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.815460 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71927f5a-bf06-4703-9470-8398d43031b0-config\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.815566 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.815566 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-certificates\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.815566 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-trusted-ca\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.815717 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-service-ca-bundle\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.815717 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h679m\" (UniqueName: \"kubernetes.io/projected/a634a756-66ac-4cd1-874a-261e048a16ef-kube-api-access-h679m\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:39.815717 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-tmp\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.815953 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71927f5a-bf06-4703-9470-8398d43031b0-trusted-ca\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.816012 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.815979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-tmp\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.816078 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.816062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71927f5a-bf06-4703-9470-8398d43031b0-config\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.816128 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.816076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-trusted-ca-bundle\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.816375 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.816354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-service-ca-bundle\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.817923 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.817906    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71927f5a-bf06-4703-9470-8398d43031b0-serving-cert\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.817999 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.817931    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-serving-cert\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.822837 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.822815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpc5x\" (UniqueName: \"kubernetes.io/projected/b8bae1e3-c38a-4e9a-a6a2-e4625a779de4-kube-api-access-tpc5x\") pod \"insights-operator-5844b856db-8lgjz\" (UID: \"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4\") " pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.822975 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.822957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r29dj\" (UniqueName: \"kubernetes.io/projected/71927f5a-bf06-4703-9470-8398d43031b0-kube-api-access-r29dj\") pod \"console-operator-696884fb9c-f9dtl\" (UID: \"71927f5a-bf06-4703-9470-8398d43031b0\") " pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:39.905875 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.905810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-8lgjz"
Sep 14 07:49:39.916058 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-installation-pull-secrets\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916164 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:39.916164 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-bound-sa-token\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916164 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916325 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.916188    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:39.916325 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.916202    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d4864867-9grlk: secret "image-registry-tls" not found
Sep 14 07:49:39.916325 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.916265    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls podName:6a0dc390-96dd-4763-889a-d7e8dd8ff645 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.416246747 +0000 UTC m=+36.124260877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls") pod "image-registry-55d4864867-9grlk" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645") : secret "image-registry-tls" not found
Sep 14 07:49:39.916439 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-certificates\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916439 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-trusted-ca\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916591 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h679m\" (UniqueName: \"kubernetes.io/projected/a634a756-66ac-4cd1-874a-261e048a16ef-kube-api-access-h679m\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:39.916591 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.916527    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:49:39.916695 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:39.916593    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls podName:a634a756-66ac-4cd1-874a-261e048a16ef nodeName:}" failed. No retries permitted until 2026-09-14 07:49:40.416575561 +0000 UTC m=+36.124589688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-txr8l" (UID: "a634a756-66ac-4cd1-874a-261e048a16ef") : secret "samples-operator-tls" not found
Sep 14 07:49:39.916695 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-image-registry-private-configuration\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916695 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a0dc390-96dd-4763-889a-d7e8dd8ff645-ca-trust-extracted\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.916892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.916714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpspx\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-kube-api-access-jpspx\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.917098 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.917078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-certificates\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.917166 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.917079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a0dc390-96dd-4763-889a-d7e8dd8ff645-ca-trust-extracted\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.917373 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.917351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-trusted-ca\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.918976 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.918957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-installation-pull-secrets\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.919080 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.919061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-image-registry-private-configuration\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.925871 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.925847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-bound-sa-token\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.925981 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.925965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h679m\" (UniqueName: \"kubernetes.io/projected/a634a756-66ac-4cd1-874a-261e048a16ef-kube-api-access-h679m\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:39.926287 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.926270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpspx\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-kube-api-access-jpspx\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:39.927976 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:39.927954    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:40.027223 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.027189    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8f02f-945b-4f0b-84bf-98a0053d7f18" containerID="ba98e41bc98cfbfa37cd34a48aa37f69e93e273eb0080d7f73eaa014852692d5" exitCode=0
Sep 14 07:49:40.027349 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.027284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerDied","Data":"ba98e41bc98cfbfa37cd34a48aa37f69e93e273eb0080d7f73eaa014852692d5"}
Sep 14 07:49:40.029631 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.029568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rzkr2" event={"ID":"97683ac6-241b-47bc-b93d-2b921a7f7140","Type":"ContainerStarted","Data":"6a7c8041f8ab91b7b500f509edc659ae4e7a7e4d4b74e0e1ccd5182899ad06e6"}
Sep 14 07:49:40.203207 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.203181    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-8lgjz"]
Sep 14 07:49:40.207300 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:40.207274    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bae1e3_c38a_4e9a_a6a2_e4625a779de4.slice/crio-e1ceac092cd2c930cd5ea70de39ae510567e174478ad4c14a955725bad879573 WatchSource:0}: Error finding container e1ceac092cd2c930cd5ea70de39ae510567e174478ad4c14a955725bad879573: Status 404 returned error can't find the container with id e1ceac092cd2c930cd5ea70de39ae510567e174478ad4c14a955725bad879573
Sep 14 07:49:40.244025 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.243999    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-f9dtl"]
Sep 14 07:49:40.257583 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:49:40.257555    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71927f5a_bf06_4703_9470_8398d43031b0.slice/crio-b2afeeec14c1b3fcecfb1522555ca1ddc8da818ecc7a3822f6ad8fb658567a6f WatchSource:0}: Error finding container b2afeeec14c1b3fcecfb1522555ca1ddc8da818ecc7a3822f6ad8fb658567a6f: Status 404 returned error can't find the container with id b2afeeec14c1b3fcecfb1522555ca1ddc8da818ecc7a3822f6ad8fb658567a6f
Sep 14 07:49:40.421014 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.420979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.421030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:40.421059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421140    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421183    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421200    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421215    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d4864867-9grlk: secret "image-registry-tls" not found
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421219    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls podName:a634a756-66ac-4cd1-874a-261e048a16ef nodeName:}" failed. No retries permitted until 2026-09-14 07:49:41.421198034 +0000 UTC m=+37.129212155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-txr8l" (UID: "a634a756-66ac-4cd1-874a-261e048a16ef") : secret "samples-operator-tls" not found
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421239    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls podName:af1713b8-c170-4d6b-a8d9-b53309b2ac75 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:42.421229237 +0000 UTC m=+38.129243360 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls") pod "dns-default-9bdwp" (UID: "af1713b8-c170-4d6b-a8d9-b53309b2ac75") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:40.421438 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:40.421256    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls podName:6a0dc390-96dd-4763-889a-d7e8dd8ff645 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:41.421247808 +0000 UTC m=+37.129261924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls") pod "image-registry-55d4864867-9grlk" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645") : secret "image-registry-tls" not found
Sep 14 07:49:41.035534 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:41.035259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-8lgjz" event={"ID":"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4","Type":"ContainerStarted","Data":"e1ceac092cd2c930cd5ea70de39ae510567e174478ad4c14a955725bad879573"}
Sep 14 07:49:41.041043 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:41.041012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5gctk" event={"ID":"21b8f02f-945b-4f0b-84bf-98a0053d7f18","Type":"ContainerStarted","Data":"0eff5c9744ae57c8fdb02401ada453890841962d3d65ed6bd785729b04b410d8"}
Sep 14 07:49:41.042642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:41.042617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" event={"ID":"71927f5a-bf06-4703-9470-8398d43031b0","Type":"ContainerStarted","Data":"b2afeeec14c1b3fcecfb1522555ca1ddc8da818ecc7a3822f6ad8fb658567a6f"}
Sep 14 07:49:41.063682 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:41.063633    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5gctk" podStartSLOduration=8.273827546 podStartE2EDuration="36.063614673s" totalImagesPullingTime="27.789787127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:41.062949778 +0000 UTC m=+36.770963910" watchObservedRunningTime="2026-09-14 07:49:41.063614673 +0000 UTC m=+36.771628814"
Sep 14 07:49:41.429531 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:41.429493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:41.430046 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:41.429559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:41.430046 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:41.429653    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:49:41.430046 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:41.429678    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:41.430046 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:41.429693    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d4864867-9grlk: secret "image-registry-tls" not found
Sep 14 07:49:41.430046 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:41.429749    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls podName:a634a756-66ac-4cd1-874a-261e048a16ef nodeName:}" failed. No retries permitted until 2026-09-14 07:49:43.429713183 +0000 UTC m=+39.137727323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-txr8l" (UID: "a634a756-66ac-4cd1-874a-261e048a16ef") : secret "samples-operator-tls" not found
Sep 14 07:49:41.430046 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:41.429773    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls podName:6a0dc390-96dd-4763-889a-d7e8dd8ff645 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:43.429762007 +0000 UTC m=+39.137776124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls") pod "image-registry-55d4864867-9grlk" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645") : secret "image-registry-tls" not found
Sep 14 07:49:42.440214 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:42.440177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:42.440646 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:42.440345    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:42.440646 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:42.440425    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls podName:af1713b8-c170-4d6b-a8d9-b53309b2ac75 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.440404515 +0000 UTC m=+42.148418631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls") pod "dns-default-9bdwp" (UID: "af1713b8-c170-4d6b-a8d9-b53309b2ac75") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:43.448573 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:43.448537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:43.448915 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:43.448582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:43.448915 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:43.448698    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:49:43.448915 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:43.448758    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:43.448915 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:43.448771    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls podName:a634a756-66ac-4cd1-874a-261e048a16ef nodeName:}" failed. No retries permitted until 2026-09-14 07:49:47.448757699 +0000 UTC m=+43.156771829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-txr8l" (UID: "a634a756-66ac-4cd1-874a-261e048a16ef") : secret "samples-operator-tls" not found
Sep 14 07:49:43.448915 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:43.448777    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d4864867-9grlk: secret "image-registry-tls" not found
Sep 14 07:49:43.448915 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:43.448820    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls podName:6a0dc390-96dd-4763-889a-d7e8dd8ff645 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:47.448806893 +0000 UTC m=+43.156821011 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls") pod "image-registry-55d4864867-9grlk" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645") : secret "image-registry-tls" not found
Sep 14 07:49:44.052429 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.052402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rzkr2" event={"ID":"97683ac6-241b-47bc-b93d-2b921a7f7140","Type":"ContainerStarted","Data":"70059543aed2e05398690230cf44d0b377e4b23223765f6f11f659731ce0e9c5"}
Sep 14 07:49:44.053854 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.053826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-8lgjz" event={"ID":"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4","Type":"ContainerStarted","Data":"ef6818b6487dbfe92604486db895a4e2f3a911a64fe59bd2025378ae160a63d2"}
Sep 14 07:49:44.055364 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.055343    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/0.log"
Sep 14 07:49:44.055460 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.055383    2563 generic.go:356] "Generic (PLEG): container finished" podID="71927f5a-bf06-4703-9470-8398d43031b0" containerID="35247ae3e2c7e45837fbd9976fd2193ad4c0e397ba6e770afe8b3a33e47fc32c" exitCode=255
Sep 14 07:49:44.055460 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.055413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" event={"ID":"71927f5a-bf06-4703-9470-8398d43031b0","Type":"ContainerDied","Data":"35247ae3e2c7e45837fbd9976fd2193ad4c0e397ba6e770afe8b3a33e47fc32c"}
Sep 14 07:49:44.055606 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.055591    2563 scope.go:117] "RemoveContainer" containerID="35247ae3e2c7e45837fbd9976fd2193ad4c0e397ba6e770afe8b3a33e47fc32c"
Sep 14 07:49:44.066441 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.066401    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rzkr2" podStartSLOduration=1.831527764 podStartE2EDuration="6.066388151s" totalImagesPullingTime="4.234860387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:44.065909282 +0000 UTC m=+39.773923424" watchObservedRunningTime="2026-09-14 07:49:44.066388151 +0000 UTC m=+39.774402289"
Sep 14 07:49:44.080834 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.080785    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-8lgjz" podStartSLOduration=1.773736001 podStartE2EDuration="5.080760902s" totalImagesPullingTime="3.307024901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:44.080661229 +0000 UTC m=+39.788675369" watchObservedRunningTime="2026-09-14 07:49:44.080760902 +0000 UTC m=+39.788775043"
Sep 14 07:49:44.693965 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.693929    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2vgb9"]
Sep 14 07:49:44.707096 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.707070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2vgb9"]
Sep 14 07:49:44.707202 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.707172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.709676 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.709653    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 07:49:44.709676 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.709670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bqkjn\""
Sep 14 07:49:44.709841 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.709689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 07:49:44.860534 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.860473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2e94682-b81a-42e1-b37d-58b15daa351c-crio-socket\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.860534 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.860514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2e94682-b81a-42e1-b37d-58b15daa351c-data-volume\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.860685 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.860537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.860685 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.860562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lmsw\" (UniqueName: \"kubernetes.io/projected/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-api-access-6lmsw\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.860685 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.860582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.961747 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.961712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.961871 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:44.961838    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:44.961906 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.961895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2e94682-b81a-42e1-b37d-58b15daa351c-crio-socket\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.961983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.961930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2e94682-b81a-42e1-b37d-58b15daa351c-data-volume\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.961983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.961958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.962043 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:44.961984    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.461959351 +0000 UTC m=+41.169973472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:44.962043 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.962016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lmsw\" (UniqueName: \"kubernetes.io/projected/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-api-access-6lmsw\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.962108 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:44.962075    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:44.962140 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:44.962127    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.462114069 +0000 UTC m=+41.170128199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:44.962176 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.962149    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2e94682-b81a-42e1-b37d-58b15daa351c-crio-socket\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.962270 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.962253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2e94682-b81a-42e1-b37d-58b15daa351c-data-volume\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:44.971024 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:44.971007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lmsw\" (UniqueName: \"kubernetes.io/projected/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-api-access-6lmsw\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:45.059426 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.059403    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:49:45.059804 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.059791    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/0.log"
Sep 14 07:49:45.059881 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.059821    2563 generic.go:356] "Generic (PLEG): container finished" podID="71927f5a-bf06-4703-9470-8398d43031b0" containerID="7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353" exitCode=255
Sep 14 07:49:45.059881 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.059848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" event={"ID":"71927f5a-bf06-4703-9470-8398d43031b0","Type":"ContainerDied","Data":"7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353"}
Sep 14 07:49:45.059983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.059908    2563 scope.go:117] "RemoveContainer" containerID="35247ae3e2c7e45837fbd9976fd2193ad4c0e397ba6e770afe8b3a33e47fc32c"
Sep 14 07:49:45.060153 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.060128    2563 scope.go:117] "RemoveContainer" containerID="7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353"
Sep 14 07:49:45.060369 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:45.060339    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-f9dtl_openshift-console-operator(71927f5a-bf06-4703-9470-8398d43031b0)\"" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" podUID="71927f5a-bf06-4703-9470-8398d43031b0"
Sep 14 07:49:45.466609 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.466573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:45.466609 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:45.466611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:45.466898 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:45.466687    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:45.466898 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:45.466756    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.466719055 +0000 UTC m=+42.174733172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:45.466898 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:45.466754    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:45.466898 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:45.466821    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:46.466804095 +0000 UTC m=+42.174818226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.063621 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:46.063594    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:49:46.063988 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:46.063923    2563 scope.go:117] "RemoveContainer" containerID="7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353"
Sep 14 07:49:46.064079 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.064060    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-f9dtl_openshift-console-operator(71927f5a-bf06-4703-9470-8398d43031b0)\"" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" podUID="71927f5a-bf06-4703-9470-8398d43031b0"
Sep 14 07:49:46.462130 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:46.462103    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j786z_fb587b36-5ad8-4a6e-bce6-9d00228fdef6/dns-node-resolver/0.log"
Sep 14 07:49:46.475575 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:46.475548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:46.475671 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:46.475581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:46.475671 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:46.475656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:46.475766 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.475684    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.475766 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.475749    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:46.475829 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.475754    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:48.475739826 +0000 UTC m=+44.183753956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:46.475829 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.475756    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:46.475894 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.475812    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:48.475796947 +0000 UTC m=+44.183811065 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:46.475894 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:46.475852    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls podName:af1713b8-c170-4d6b-a8d9-b53309b2ac75 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:54.475843624 +0000 UTC m=+50.183857744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls") pod "dns-default-9bdwp" (UID: "af1713b8-c170-4d6b-a8d9-b53309b2ac75") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:47.462654 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:47.462629    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2jjqz_32354ba1-68bb-449a-88b5-629a449b78be/node-ca/0.log"
Sep 14 07:49:47.483864 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:47.483833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:47.483959 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:47.483875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:47.484003 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:47.483984    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:49:47.484061 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:47.484045    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:47.484099 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:47.484065    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d4864867-9grlk: secret "image-registry-tls" not found
Sep 14 07:49:47.484129 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:47.484054    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls podName:a634a756-66ac-4cd1-874a-261e048a16ef nodeName:}" failed. No retries permitted until 2026-09-14 07:49:55.484037539 +0000 UTC m=+51.192051664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-txr8l" (UID: "a634a756-66ac-4cd1-874a-261e048a16ef") : secret "samples-operator-tls" not found
Sep 14 07:49:47.484168 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:47.484128    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls podName:6a0dc390-96dd-4763-889a-d7e8dd8ff645 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:55.484115347 +0000 UTC m=+51.192129469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls") pod "image-registry-55d4864867-9grlk" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645") : secret "image-registry-tls" not found
Sep 14 07:49:48.490096 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:48.490058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:48.490096 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:48.490112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:48.490580 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:48.490221    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:48.490580 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:48.490259    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:48.490580 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:48.490289    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:52.490274395 +0000 UTC m=+48.198288512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:48.490580 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:48.490310    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:49:52.490297893 +0000 UTC m=+48.198312045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:48.662114 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:48.662066    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rzkr2_97683ac6-241b-47bc-b93d-2b921a7f7140/serve-healthcheck-canary/0.log"
Sep 14 07:49:49.928939 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:49.928902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:49.929393 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:49.928985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:49:49.929393 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:49.929354    2563 scope.go:117] "RemoveContainer" containerID="7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353"
Sep 14 07:49:49.929559 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:49.929537    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-f9dtl_openshift-console-operator(71927f5a-bf06-4703-9470-8398d43031b0)\"" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" podUID="71927f5a-bf06-4703-9470-8398d43031b0"
Sep 14 07:49:50.072578 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:50.072550    2563 scope.go:117] "RemoveContainer" containerID="7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353"
Sep 14 07:49:50.072743 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:50.072689    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-f9dtl_openshift-console-operator(71927f5a-bf06-4703-9470-8398d43031b0)\"" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" podUID="71927f5a-bf06-4703-9470-8398d43031b0"
Sep 14 07:49:52.519707 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:52.519672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:52.519707 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:52.519710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:49:52.520152 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:52.519803    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:52.520152 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:52.519832    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:52.520152 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:52.519855    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:50:00.519841573 +0000 UTC m=+56.227855690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:52.520152 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:52.519901    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls podName:f2e94682-b81a-42e1-b37d-58b15daa351c nodeName:}" failed. No retries permitted until 2026-09-14 07:50:00.519887296 +0000 UTC m=+56.227901429 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2vgb9" (UID: "f2e94682-b81a-42e1-b37d-58b15daa351c") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:54.533826 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:54.533781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:49:54.534245 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:54.533931    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:54.534245 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:54.533997    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls podName:af1713b8-c170-4d6b-a8d9-b53309b2ac75 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:10.533981772 +0000 UTC m=+66.241995892 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls") pod "dns-default-9bdwp" (UID: "af1713b8-c170-4d6b-a8d9-b53309b2ac75") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:55.541892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:55.541855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:49:55.542284 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:49:55.541900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:49:55.542284 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:55.541999    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:49:55.542284 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:55.542018    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:55.542284 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:55.542028    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55d4864867-9grlk: secret "image-registry-tls" not found
Sep 14 07:49:55.542284 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:55.542077    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls podName:6a0dc390-96dd-4763-889a-d7e8dd8ff645 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.542061742 +0000 UTC m=+67.250075859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls") pod "image-registry-55d4864867-9grlk" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645") : secret "image-registry-tls" not found
Sep 14 07:49:55.542284 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:49:55.542092    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls podName:a634a756-66ac-4cd1-874a-261e048a16ef nodeName:}" failed. No retries permitted until 2026-09-14 07:50:11.542085162 +0000 UTC m=+67.250099279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-txr8l" (UID: "a634a756-66ac-4cd1-874a-261e048a16ef") : secret "samples-operator-tls" not found
Sep 14 07:50:00.583976 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:00.583939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:50:00.583976 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:00.583977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:50:00.584508 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:00.584456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2e94682-b81a-42e1-b37d-58b15daa351c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:50:00.586176 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:00.586156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2e94682-b81a-42e1-b37d-58b15daa351c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2vgb9\" (UID: \"f2e94682-b81a-42e1-b37d-58b15daa351c\") " pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:50:00.616177 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:00.616156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2vgb9"
Sep 14 07:50:00.884371 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:00.884287    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2vgb9"]
Sep 14 07:50:00.888277 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:00.888247    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e94682_b81a_42e1_b37d_58b15daa351c.slice/crio-7d3ecf457bf05cd29248acf1b1d96d1b975f6c35a55d622f1e469e75c781f55f WatchSource:0}: Error finding container 7d3ecf457bf05cd29248acf1b1d96d1b975f6c35a55d622f1e469e75c781f55f: Status 404 returned error can't find the container with id 7d3ecf457bf05cd29248acf1b1d96d1b975f6c35a55d622f1e469e75c781f55f
Sep 14 07:50:01.099330 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:01.099296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vgb9" event={"ID":"f2e94682-b81a-42e1-b37d-58b15daa351c","Type":"ContainerStarted","Data":"7a6592d3225edba5d9607ede4bad4605e6b526ef7b34d64fa325dda31408f735"}
Sep 14 07:50:01.099465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:01.099351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vgb9" event={"ID":"f2e94682-b81a-42e1-b37d-58b15daa351c","Type":"ContainerStarted","Data":"7d3ecf457bf05cd29248acf1b1d96d1b975f6c35a55d622f1e469e75c781f55f"}
Sep 14 07:50:03.106251 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:03.106215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vgb9" event={"ID":"f2e94682-b81a-42e1-b37d-58b15daa351c","Type":"ContainerStarted","Data":"307803129ad62b55f2b11b4f42c035e972ddc28349723e28d919b6e3a661d634"}
Sep 14 07:50:04.010339 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:04.010274    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hl96f"
Sep 14 07:50:04.111737 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:04.111701    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2vgb9" event={"ID":"f2e94682-b81a-42e1-b37d-58b15daa351c","Type":"ContainerStarted","Data":"97d9e741210932690c412c52f3846745e8eb779cb0594b390d818b8e4d720142"}
Sep 14 07:50:04.127598 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:04.127555    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2vgb9" podStartSLOduration=17.456509543 podStartE2EDuration="20.127541194s" totalImagesPullingTime="2.671031651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:04.127406492 +0000 UTC m=+59.835420622" watchObservedRunningTime="2026-09-14 07:50:04.127541194 +0000 UTC m=+59.835555333"
Sep 14 07:50:04.785447 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:04.785422    2563 scope.go:117] "RemoveContainer" containerID="7041d991b0f63e6d61c5ac415458077e3f069e3b1403edf4b175ca6fc2a3b353"
Sep 14 07:50:05.117546 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:05.117465    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:50:05.117546 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:05.117521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" event={"ID":"71927f5a-bf06-4703-9470-8398d43031b0","Type":"ContainerStarted","Data":"1416211d9aec8c7277217b8f044e342b2807d71eadc29664772b41474ab71a29"}
Sep 14 07:50:05.118014 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:05.117951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:50:05.134196 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:05.134142    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl" podStartSLOduration=22.871290766 podStartE2EDuration="26.134130918s" totalImagesPullingTime="3.262840152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:05.133436431 +0000 UTC m=+60.841450570" watchObservedRunningTime="2026-09-14 07:50:05.134130918 +0000 UTC m=+60.842145053"
Sep 14 07:50:05.702211 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:05.702169    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-f9dtl"
Sep 14 07:50:10.452947 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.452912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:50:10.455121 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.455100    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:50:10.465963 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.465943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24-original-pull-secret\") pod \"global-pull-secret-syncer-cxxtl\" (UID: \"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24\") " pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:50:10.553956 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.553923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:50:10.554093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.553969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:50:10.554093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.554023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:50:10.556295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.556266    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:50:10.556396 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.556294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/af1713b8-c170-4d6b-a8d9-b53309b2ac75-metrics-tls\") pod \"dns-default-9bdwp\" (UID: \"af1713b8-c170-4d6b-a8d9-b53309b2ac75\") " pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:50:10.556983 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.556968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:50:10.566396 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.566373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/223808bc-d985-4ead-8463-c63a3972b62b-metrics-certs\") pod \"network-metrics-daemon-7xmvb\" (UID: \"223808bc-d985-4ead-8463-c63a3972b62b\") " pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:50:10.566583 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.566571    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:50:10.577227 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.577205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k65zl\" (UniqueName: \"kubernetes.io/projected/eff243d2-766c-46ab-9fdb-d8c8e4fabb4a-kube-api-access-k65zl\") pod \"network-check-target-s7jl4\" (UID: \"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a\") " pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:50:10.595335 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.595316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfw6g\""
Sep 14 07:50:10.600449 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.600431    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wkwvc\""
Sep 14 07:50:10.604206 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.604192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7xmvb"
Sep 14 07:50:10.605370 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.605355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cxxtl"
Sep 14 07:50:10.608628 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.608613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:50:10.770923 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.770847    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gsd8p\""
Sep 14 07:50:10.778791 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.778769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:50:10.885373 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.885350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cxxtl"]
Sep 14 07:50:10.888391 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:10.888322    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f884ddb_b97a_4fe4_bd3a_a0aaeb6e0d24.slice/crio-60ac6d5e8ee3bab279b016cb15d954a527b44d646725ff709e5c6d222df610c3 WatchSource:0}: Error finding container 60ac6d5e8ee3bab279b016cb15d954a527b44d646725ff709e5c6d222df610c3: Status 404 returned error can't find the container with id 60ac6d5e8ee3bab279b016cb15d954a527b44d646725ff709e5c6d222df610c3
Sep 14 07:50:10.890023 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.889937    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7xmvb"]
Sep 14 07:50:10.891005 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:10.890971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s7jl4"]
Sep 14 07:50:10.892762 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:10.892738    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223808bc_d985_4ead_8463_c63a3972b62b.slice/crio-cb50511bbe4ea142b96a2a683ef6b8f7554868932d9cc0307dd884b5bc7b72ca WatchSource:0}: Error finding container cb50511bbe4ea142b96a2a683ef6b8f7554868932d9cc0307dd884b5bc7b72ca: Status 404 returned error can't find the container with id cb50511bbe4ea142b96a2a683ef6b8f7554868932d9cc0307dd884b5bc7b72ca
Sep 14 07:50:10.893595 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:10.893576    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff243d2_766c_46ab_9fdb_d8c8e4fabb4a.slice/crio-abefbb4a6c8dda34033ce9c0ee21e1d84c2f6865399cccc789e357d209399f3c WatchSource:0}: Error finding container abefbb4a6c8dda34033ce9c0ee21e1d84c2f6865399cccc789e357d209399f3c: Status 404 returned error can't find the container with id abefbb4a6c8dda34033ce9c0ee21e1d84c2f6865399cccc789e357d209399f3c
Sep 14 07:50:11.046440 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.046372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9bdwp"]
Sep 14 07:50:11.049249 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:11.049229    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1713b8_c170_4d6b_a8d9_b53309b2ac75.slice/crio-273751df8561a9200b225c30e35f903c91a9cfe88a63ae6956d8fb6c498a6752 WatchSource:0}: Error finding container 273751df8561a9200b225c30e35f903c91a9cfe88a63ae6956d8fb6c498a6752: Status 404 returned error can't find the container with id 273751df8561a9200b225c30e35f903c91a9cfe88a63ae6956d8fb6c498a6752
Sep 14 07:50:11.136693 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.136658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xmvb" event={"ID":"223808bc-d985-4ead-8463-c63a3972b62b","Type":"ContainerStarted","Data":"cb50511bbe4ea142b96a2a683ef6b8f7554868932d9cc0307dd884b5bc7b72ca"}
Sep 14 07:50:11.137795 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.137769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cxxtl" event={"ID":"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24","Type":"ContainerStarted","Data":"60ac6d5e8ee3bab279b016cb15d954a527b44d646725ff709e5c6d222df610c3"}
Sep 14 07:50:11.138846 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.138822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s7jl4" event={"ID":"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a","Type":"ContainerStarted","Data":"abefbb4a6c8dda34033ce9c0ee21e1d84c2f6865399cccc789e357d209399f3c"}
Sep 14 07:50:11.139909 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.139890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bdwp" event={"ID":"af1713b8-c170-4d6b-a8d9-b53309b2ac75","Type":"ContainerStarted","Data":"273751df8561a9200b225c30e35f903c91a9cfe88a63ae6956d8fb6c498a6752"}
Sep 14 07:50:11.562110 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.562069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:50:11.562643 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.562132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:11.566506 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.566478    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"image-registry-55d4864867-9grlk\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") " pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:11.568023 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.567584    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a634a756-66ac-4cd1-874a-261e048a16ef-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-txr8l\" (UID: \"a634a756-66ac-4cd1-874a-261e048a16ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:50:11.807280 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.807219    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-flrlx\""
Sep 14 07:50:11.814777 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.814690    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"
Sep 14 07:50:11.817423 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.817237    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfxc9\""
Sep 14 07:50:11.825589 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:11.825354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:12.146982 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.146939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xmvb" event={"ID":"223808bc-d985-4ead-8463-c63a3972b62b","Type":"ContainerStarted","Data":"1881e3edd79674335ee126d7bbfdbd3d8ddac37cde4110c2d1ce362105e421d3"}
Sep 14 07:50:12.177807 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.177761    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l"]
Sep 14 07:50:12.194552 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.187355    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55d4864867-9grlk"]
Sep 14 07:50:12.206332 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:12.206300    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0dc390_96dd_4763_889a_d7e8dd8ff645.slice/crio-724c8864fc4ebb5a07ee1034e9130cf73f6916cd82c5f60da00ad473d2255329 WatchSource:0}: Error finding container 724c8864fc4ebb5a07ee1034e9130cf73f6916cd82c5f60da00ad473d2255329: Status 404 returned error can't find the container with id 724c8864fc4ebb5a07ee1034e9130cf73f6916cd82c5f60da00ad473d2255329
Sep 14 07:50:12.433425 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.433395    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d4864867-9grlk"]
Sep 14 07:50:12.452370 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.452271    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"]
Sep 14 07:50:12.457684 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.456457    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.459740 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.459505    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:50:12.462194 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.459898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 14 07:50:12.462194 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.460005    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:50:12.462194 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.460015    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:50:12.480456 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.480386    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"]
Sep 14 07:50:12.550411 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.550065    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-fblrm"]
Sep 14 07:50:12.555533 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.554568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:12.557886 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.557635    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5nl8j\""
Sep 14 07:50:12.558279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.558151    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 07:50:12.558279 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.558156    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 07:50:12.569524 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.569483    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-fblrm"]
Sep 14 07:50:12.573195 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.573046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwswm\" (UniqueName: \"kubernetes.io/projected/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-kube-api-access-xwswm\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.573548 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.573502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.573647 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.573580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-tmp\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.674820 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.674247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwswm\" (UniqueName: \"kubernetes.io/projected/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-kube-api-access-xwswm\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.674820 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.674299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.674820 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.674346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-tmp\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.674820 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.674386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9m4m\" (UniqueName: \"kubernetes.io/projected/9fe2ba75-cb96-46a8-8a04-321885ea1cc7-kube-api-access-j9m4m\") pod \"downloads-76f99584f4-fblrm\" (UID: \"9fe2ba75-cb96-46a8-8a04-321885ea1cc7\") " pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:12.675491 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.675448    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-tmp\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.678556 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.678510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.687805 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.686867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwswm\" (UniqueName: \"kubernetes.io/projected/8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2-kube-api-access-xwswm\") pod \"klusterlet-addon-workmgr-6d5c6b86f-vf496\" (UID: \"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.775706 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.775661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9m4m\" (UniqueName: \"kubernetes.io/projected/9fe2ba75-cb96-46a8-8a04-321885ea1cc7-kube-api-access-j9m4m\") pod \"downloads-76f99584f4-fblrm\" (UID: \"9fe2ba75-cb96-46a8-8a04-321885ea1cc7\") " pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:12.788988 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.788922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9m4m\" (UniqueName: \"kubernetes.io/projected/9fe2ba75-cb96-46a8-8a04-321885ea1cc7-kube-api-access-j9m4m\") pod \"downloads-76f99584f4-fblrm\" (UID: \"9fe2ba75-cb96-46a8-8a04-321885ea1cc7\") " pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:12.791573 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.791544    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:12.866311 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:12.866174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:13.155568 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.155483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l" event={"ID":"a634a756-66ac-4cd1-874a-261e048a16ef","Type":"ContainerStarted","Data":"57ef38a1342f236717f0c862c78be5ab5a2243457b5cb01cde2216503f1bfd0e"}
Sep 14 07:50:13.158636 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.158601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7xmvb" event={"ID":"223808bc-d985-4ead-8463-c63a3972b62b","Type":"ContainerStarted","Data":"e9e7f1d59755228bd8b38123d0d8c953f4cf0e73f1e8a1138ee5f748b55547ab"}
Sep 14 07:50:13.161663 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.161634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d4864867-9grlk" event={"ID":"6a0dc390-96dd-4763-889a-d7e8dd8ff645","Type":"ContainerStarted","Data":"c3ab6b5f93b5a784a40e8329223f759599a129b613f0659f6332b523c538ac95"}
Sep 14 07:50:13.161792 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.161684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d4864867-9grlk" event={"ID":"6a0dc390-96dd-4763-889a-d7e8dd8ff645","Type":"ContainerStarted","Data":"724c8864fc4ebb5a07ee1034e9130cf73f6916cd82c5f60da00ad473d2255329"}
Sep 14 07:50:13.162092 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.162076    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:13.197657 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.197598    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55d4864867-9grlk" podStartSLOduration=34.197579769 podStartE2EDuration="34.197579769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:13.197072901 +0000 UTC m=+68.905087039" watchObservedRunningTime="2026-09-14 07:50:13.197579769 +0000 UTC m=+68.905593910"
Sep 14 07:50:13.198277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.198229    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7xmvb" podStartSLOduration=67.149002435 podStartE2EDuration="1m8.198213451s" totalImagesPullingTime="1.049211016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:13.179195243 +0000 UTC m=+68.887209383" watchObservedRunningTime="2026-09-14 07:50:13.198213451 +0000 UTC m=+68.906227570"
Sep 14 07:50:13.491944 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.491413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-fblrm"]
Sep 14 07:50:13.496951 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:13.496921    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe2ba75_cb96_46a8_8a04_321885ea1cc7.slice/crio-018ea95d2fb4c8165f1991128e2fc261dd478cdc80b1c9bce71bd4e80b5c409a WatchSource:0}: Error finding container 018ea95d2fb4c8165f1991128e2fc261dd478cdc80b1c9bce71bd4e80b5c409a: Status 404 returned error can't find the container with id 018ea95d2fb4c8165f1991128e2fc261dd478cdc80b1c9bce71bd4e80b5c409a
Sep 14 07:50:13.501458 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:13.501437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"]
Sep 14 07:50:14.167564 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.167521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-fblrm" event={"ID":"9fe2ba75-cb96-46a8-8a04-321885ea1cc7","Type":"ContainerStarted","Data":"018ea95d2fb4c8165f1991128e2fc261dd478cdc80b1c9bce71bd4e80b5c409a"}
Sep 14 07:50:14.170277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.169895    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bdwp" event={"ID":"af1713b8-c170-4d6b-a8d9-b53309b2ac75","Type":"ContainerStarted","Data":"86a06ec05434f15a95ac3c04ffa0683017362ed516619e96806a699667939271"}
Sep 14 07:50:14.170277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.169953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9bdwp" event={"ID":"af1713b8-c170-4d6b-a8d9-b53309b2ac75","Type":"ContainerStarted","Data":"57ae2fde6093a16bc8f2a642013a91d1feb74f7bbf40561ca6ae49c48f9018c1"}
Sep 14 07:50:14.170277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.170054    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:50:14.171618 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.171581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496" event={"ID":"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2","Type":"ContainerStarted","Data":"aa845c3ab1e47c31f178f03e4df498d682269d97d7db178c1b15ee5ad6c73b53"}
Sep 14 07:50:14.188887 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.188525    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9bdwp" podStartSLOduration=33.97393895 podStartE2EDuration="36.188503849s" totalImagesPullingTime="2.214564899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:14.186931516 +0000 UTC m=+69.894945664" watchObservedRunningTime="2026-09-14 07:50:14.188503849 +0000 UTC m=+69.896518001"
Sep 14 07:50:14.404057 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.404027    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9f5589f5-scz9n"]
Sep 14 07:50:14.407103 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.407080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.409964 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.409938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 07:50:14.410081 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.409989    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 07:50:14.410179 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.410155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hvfj5\""
Sep 14 07:50:14.410441 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.410421    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 07:50:14.410515 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.410464    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 07:50:14.410566 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.410467    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 07:50:14.418075 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.418012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9f5589f5-scz9n"]
Sep 14 07:50:14.491320 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.491285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-oauth-config\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.491450 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.491354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8vvn\" (UniqueName: \"kubernetes.io/projected/32169d13-b2ef-467d-a0a9-049de18fcced-kube-api-access-g8vvn\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.491450 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.491378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-console-config\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.491450 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.491401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-service-ca\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.491450 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.491427    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-serving-cert\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.491639 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.491491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-oauth-serving-cert\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.592846 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.592810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-console-config\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593019 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.592853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-service-ca\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593019 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.592879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-serving-cert\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593019 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.592911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-oauth-serving-cert\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593168 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.593022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-oauth-config\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593168 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.593078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8vvn\" (UniqueName: \"kubernetes.io/projected/32169d13-b2ef-467d-a0a9-049de18fcced-kube-api-access-g8vvn\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593645 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.593619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-console-config\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.593967 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.593925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-service-ca\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.594251 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.594219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-oauth-serving-cert\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.595820 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.595794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-oauth-config\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.595940 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.595918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-serving-cert\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.608321 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.608297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8vvn\" (UniqueName: \"kubernetes.io/projected/32169d13-b2ef-467d-a0a9-049de18fcced-kube-api-access-g8vvn\") pod \"console-9f5589f5-scz9n\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") " pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:14.718169 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:14.718139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:17.557376 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:17.557072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9f5589f5-scz9n"]
Sep 14 07:50:18.190498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.189960    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cxxtl" event={"ID":"0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24","Type":"ContainerStarted","Data":"87ab23f702714eac3ee4dd9b424b9874a927d6ab672c6a3bf8e6ba5898832713"}
Sep 14 07:50:18.192687 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.192624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5589f5-scz9n" event={"ID":"32169d13-b2ef-467d-a0a9-049de18fcced","Type":"ContainerStarted","Data":"512aa15a5fa23b0bb6a763d5906a2ee29ad330e3ac6a6f2345619a8559d47dbe"}
Sep 14 07:50:18.195558 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.195047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s7jl4" event={"ID":"eff243d2-766c-46ab-9fdb-d8c8e4fabb4a","Type":"ContainerStarted","Data":"ccdf81f8339cb8c2620524ae801fe250fd9d78727178b57740d702a9f6fb6432"}
Sep 14 07:50:18.195558 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.195523    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:50:18.198410 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.198196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l" event={"ID":"a634a756-66ac-4cd1-874a-261e048a16ef","Type":"ContainerStarted","Data":"62adef7b1e51acec542f6028bdfe540cb395f7ee91dcd79e71c187bf96e6f2fb"}
Sep 14 07:50:18.198410 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.198246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l" event={"ID":"a634a756-66ac-4cd1-874a-261e048a16ef","Type":"ContainerStarted","Data":"ec498b9d3fdceb4893009d5da7209d28d2e85eb9c07fd914a7d7067759f2cb4a"}
Sep 14 07:50:18.204930 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.204858    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cxxtl" podStartSLOduration=66.821384327 podStartE2EDuration="1m13.204841108s" totalImagesPullingTime="6.383456781s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:18.204074691 +0000 UTC m=+73.912088844" watchObservedRunningTime="2026-09-14 07:50:18.204841108 +0000 UTC m=+73.912855249"
Sep 14 07:50:18.238793 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.237615    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-txr8l" podStartSLOduration=34.200614252 podStartE2EDuration="39.237595567s" totalImagesPullingTime="5.036981315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:18.22133664 +0000 UTC m=+73.929350780" watchObservedRunningTime="2026-09-14 07:50:18.237595567 +0000 UTC m=+73.945609707"
Sep 14 07:50:18.238793 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:18.237937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s7jl4" podStartSLOduration=66.860674948 podStartE2EDuration="1m13.237928792s" totalImagesPullingTime="6.377253844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:18.237054407 +0000 UTC m=+73.945068608" watchObservedRunningTime="2026-09-14 07:50:18.237928792 +0000 UTC m=+73.945942932"
Sep 14 07:50:19.187418 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.187377    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56bcf48cd-px48q"]
Sep 14 07:50:19.196195 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.196168    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.200570 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.200180    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56bcf48cd-px48q"]
Sep 14 07:50:19.206855 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.205858    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 07:50:19.334337 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-oauth-config\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.334524 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-oauth-serving-cert\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.334524 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-service-ca\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.334524 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqpbn\" (UniqueName: \"kubernetes.io/projected/b02de419-bff6-44c3-90ab-5d47e38ef82d-kube-api-access-jqpbn\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.334524 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-trusted-ca-bundle\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.334756 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-serving-cert\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.334756 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.334562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-config\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435212 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435172    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-config\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435388 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-oauth-config\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435388 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-oauth-serving-cert\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435506 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-service-ca\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435569 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqpbn\" (UniqueName: \"kubernetes.io/projected/b02de419-bff6-44c3-90ab-5d47e38ef82d-kube-api-access-jqpbn\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435569 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-trusted-ca-bundle\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.435673 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.435599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-serving-cert\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.436129 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.436050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-config\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.436406 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.436377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-service-ca\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.436507 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.436416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-oauth-serving-cert\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.436884 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.436839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-trusted-ca-bundle\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.439796 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.438620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-serving-cert\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.440379 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.440357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-oauth-config\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.443933 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.443892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqpbn\" (UniqueName: \"kubernetes.io/projected/b02de419-bff6-44c3-90ab-5d47e38ef82d-kube-api-access-jqpbn\") pod \"console-56bcf48cd-px48q\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") " pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.509918 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.509677    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:19.859601 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.859500    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"]
Sep 14 07:50:19.861685 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.861658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:19.863910 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.863881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 14 07:50:19.864032 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.863929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s4kks\""
Sep 14 07:50:19.874541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.874494    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"]
Sep 14 07:50:19.940767 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:19.940712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b2f954db-a555-4515-8dd8-29492c52b79b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-8fkqq\" (UID: \"b2f954db-a555-4515-8dd8-29492c52b79b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:20.042123 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:20.042081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b2f954db-a555-4515-8dd8-29492c52b79b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-8fkqq\" (UID: \"b2f954db-a555-4515-8dd8-29492c52b79b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:20.045056 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:20.045027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b2f954db-a555-4515-8dd8-29492c52b79b-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-8fkqq\" (UID: \"b2f954db-a555-4515-8dd8-29492c52b79b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:20.175987 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:20.175905    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:22.194076 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.194052    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"]
Sep 14 07:50:22.196790 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:22.196756    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f954db_a555_4515_8dd8_29492c52b79b.slice/crio-65d7cbc1ddd36e0bf509304e53edcdaf0c87b59979d3cfba46d3e2bf5a77a912 WatchSource:0}: Error finding container 65d7cbc1ddd36e0bf509304e53edcdaf0c87b59979d3cfba46d3e2bf5a77a912: Status 404 returned error can't find the container with id 65d7cbc1ddd36e0bf509304e53edcdaf0c87b59979d3cfba46d3e2bf5a77a912
Sep 14 07:50:22.197969 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.197939    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56bcf48cd-px48q"]
Sep 14 07:50:22.199996 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:22.199944    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb02de419_bff6_44c3_90ab_5d47e38ef82d.slice/crio-2768a6a0c28e8b872ba87b1ef8615df1fc5e19c322b94bf3d9923655cde1960f WatchSource:0}: Error finding container 2768a6a0c28e8b872ba87b1ef8615df1fc5e19c322b94bf3d9923655cde1960f: Status 404 returned error can't find the container with id 2768a6a0c28e8b872ba87b1ef8615df1fc5e19c322b94bf3d9923655cde1960f
Sep 14 07:50:22.218328 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.218060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5589f5-scz9n" event={"ID":"32169d13-b2ef-467d-a0a9-049de18fcced","Type":"ContainerStarted","Data":"74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f"}
Sep 14 07:50:22.219584 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.219558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq" event={"ID":"b2f954db-a555-4515-8dd8-29492c52b79b","Type":"ContainerStarted","Data":"65d7cbc1ddd36e0bf509304e53edcdaf0c87b59979d3cfba46d3e2bf5a77a912"}
Sep 14 07:50:22.221129 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.221081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcf48cd-px48q" event={"ID":"b02de419-bff6-44c3-90ab-5d47e38ef82d","Type":"ContainerStarted","Data":"2768a6a0c28e8b872ba87b1ef8615df1fc5e19c322b94bf3d9923655cde1960f"}
Sep 14 07:50:22.222702 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.222681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496" event={"ID":"8a1dfad0-d3df-41b6-aa31-4c2fc4f7b1d2","Type":"ContainerStarted","Data":"11fd241d44154e98f6449b3ae9db5eebf7d6335dc39ce97dd92f50d477e52b64"}
Sep 14 07:50:22.222937 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.222916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:22.224857 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.224839    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496"
Sep 14 07:50:22.233639 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.233341    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9f5589f5-scz9n" podStartSLOduration=3.873235369 podStartE2EDuration="8.233317403s" totalImagesPullingTime="4.360082034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:22.233161411 +0000 UTC m=+77.941175550" watchObservedRunningTime="2026-09-14 07:50:22.233317403 +0000 UTC m=+77.941331543"
Sep 14 07:50:22.247994 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:22.247953    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d5c6b86f-vf496" podStartSLOduration=1.831799233 podStartE2EDuration="10.247934578s" totalImagesPullingTime="8.416135345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:22.247391957 +0000 UTC m=+77.955406099" watchObservedRunningTime="2026-09-14 07:50:22.247934578 +0000 UTC m=+77.955948718"
Sep 14 07:50:23.230516 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:23.230352    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcf48cd-px48q" event={"ID":"b02de419-bff6-44c3-90ab-5d47e38ef82d","Type":"ContainerStarted","Data":"2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040"}
Sep 14 07:50:23.251008 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:23.250952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56bcf48cd-px48q" podStartSLOduration=4.250934242 podStartE2EDuration="4.250934242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:23.248890633 +0000 UTC m=+78.956904776" watchObservedRunningTime="2026-09-14 07:50:23.250934242 +0000 UTC m=+78.958948374"
Sep 14 07:50:24.177550 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.177521    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9bdwp"
Sep 14 07:50:24.235221 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.235171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq" event={"ID":"b2f954db-a555-4515-8dd8-29492c52b79b","Type":"ContainerStarted","Data":"1e99699622c3d79f04aa667ee2cfccc02767723ae57eaec5bf555db926f40397"}
Sep 14 07:50:24.235685 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.235549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:24.240590 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.240554    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq"
Sep 14 07:50:24.251924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.251874    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-8fkqq" podStartSLOduration=4.130095492 podStartE2EDuration="5.251858683s" totalImagesPullingTime="1.121763191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:24.251117863 +0000 UTC m=+79.959132003" watchObservedRunningTime="2026-09-14 07:50:24.251858683 +0000 UTC m=+79.959872821"
Sep 14 07:50:24.718313 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.718284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:24.718501 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.718346    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:24.720162 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.720122    2563 patch_prober.go:28] interesting pod/console-9f5589f5-scz9n container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused" start-of-body=
Sep 14 07:50:24.720286 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:24.720180    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-9f5589f5-scz9n" podUID="32169d13-b2ef-467d-a0a9-049de18fcced" containerName="console" probeResult="failure" output="Get \"https://10.133.0.15:8443/health\": dial tcp 10.133.0.15:8443: connect: connection refused"
Sep 14 07:50:29.510492 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:29.510348    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:29.510492 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:29.510422    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:29.516566 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:29.516415    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:30.264833 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.264800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:50:30.268168 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.268140    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"]
Sep 14 07:50:30.274880 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.274856    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.277209 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.277157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 14 07:50:30.277318 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.277282    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fwhfj\""
Sep 14 07:50:30.277378 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.277316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 14 07:50:30.277433 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.277399    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 07:50:30.278043 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.278026    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 07:50:30.278132 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.278029    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 07:50:30.283895 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.283462    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"]
Sep 14 07:50:30.315667 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.314580    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q8t8p"]
Sep 14 07:50:30.317845 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.317824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.319886 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.319851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 07:50:30.320428 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.320397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 07:50:30.320554 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.320493    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcz9h\""
Sep 14 07:50:30.320630 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.320549    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 07:50:30.325139 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.325074    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9f5589f5-scz9n"]
Sep 14 07:50:30.431705 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-textfile\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.431705 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ad65499-1979-4585-b1da-a643541ee2ac-metrics-client-ca\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.431950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.431950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-tls\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.431950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znzlq\" (UniqueName: \"kubernetes.io/projected/e2b3c120-cc4b-4797-90f8-d6233738cc21-kube-api-access-znzlq\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.431950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-sys\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.431950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2b3c120-cc4b-4797-90f8-d6233738cc21-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.431950 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.431931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b3c120-cc4b-4797-90f8-d6233738cc21-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.432252 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.432000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgsrw\" (UniqueName: \"kubernetes.io/projected/5ad65499-1979-4585-b1da-a643541ee2ac-kube-api-access-jgsrw\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.432252 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.432044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2b3c120-cc4b-4797-90f8-d6233738cc21-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.432252 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.432078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-root\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.432252 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.432113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-wtmp\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.432252 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.432140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533402 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgsrw\" (UniqueName: \"kubernetes.io/projected/5ad65499-1979-4585-b1da-a643541ee2ac-kube-api-access-jgsrw\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533402 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2b3c120-cc4b-4797-90f8-d6233738cc21-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-root\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-wtmp\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-textfile\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ad65499-1979-4585-b1da-a643541ee2ac-metrics-client-ca\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533529    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-tls\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znzlq\" (UniqueName: \"kubernetes.io/projected/e2b3c120-cc4b-4797-90f8-d6233738cc21-kube-api-access-znzlq\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-sys\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2b3c120-cc4b-4797-90f8-d6233738cc21-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-wtmp\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b3c120-cc4b-4797-90f8-d6233738cc21-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533747    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-root\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.533905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.533879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-textfile\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.534593 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.534094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ad65499-1979-4585-b1da-a643541ee2ac-sys\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.534593 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.534377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ad65499-1979-4585-b1da-a643541ee2ac-metrics-client-ca\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.534593 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.534475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2b3c120-cc4b-4797-90f8-d6233738cc21-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.534862 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.534833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.536295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.536276    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2b3c120-cc4b-4797-90f8-d6233738cc21-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.536895 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.536869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-tls\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.536895 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.536886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ad65499-1979-4585-b1da-a643541ee2ac-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.537245 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.537228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e2b3c120-cc4b-4797-90f8-d6233738cc21-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.540852 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.540834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgsrw\" (UniqueName: \"kubernetes.io/projected/5ad65499-1979-4585-b1da-a643541ee2ac-kube-api-access-jgsrw\") pod \"node-exporter-q8t8p\" (UID: \"5ad65499-1979-4585-b1da-a643541ee2ac\") " pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:30.541577 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.541555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znzlq\" (UniqueName: \"kubernetes.io/projected/e2b3c120-cc4b-4797-90f8-d6233738cc21-kube-api-access-znzlq\") pod \"openshift-state-metrics-67dcf8c979-djrd2\" (UID: \"e2b3c120-cc4b-4797-90f8-d6233738cc21\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.586322 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.586292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"
Sep 14 07:50:30.631285 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:30.631257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8t8p"
Sep 14 07:50:31.373619 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.373585    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:50:31.415676 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.415643    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:50:31.415850 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.415789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.418942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.418748    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 14 07:50:31.418942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.418765    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 14 07:50:31.418942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.418776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 14 07:50:31.418942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.418820    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 14 07:50:31.418942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.418769    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 14 07:50:31.419276 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.418996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 14 07:50:31.419276 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.419003    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 14 07:50:31.419276 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.419001    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pq7w4\""
Sep 14 07:50:31.419276 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.419273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 14 07:50:31.419467 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.419314    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 14 07:50:31.542848 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.542807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.542858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.542974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543601 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-config-out\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543601 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543601 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-web-config\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.543601 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.543366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nklh4\" (UniqueName: \"kubernetes.io/projected/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-kube-api-access-nklh4\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644295 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644314    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644485 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-config-out\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-web-config\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644592    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nklh4\" (UniqueName: \"kubernetes.io/projected/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-kube-api-access-nklh4\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.644668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.644781 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:50:31.644742    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 14 07:50:31.645052 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:50:31.644830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-main-tls podName:b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d nodeName:}" failed. No retries permitted until 2026-09-14 07:50:32.14480622 +0000 UTC m=+87.852820356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d") : secret "alertmanager-main-tls" not found
Sep 14 07:50:31.645436 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.645404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.645560 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.645531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.645801 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.645771    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.647627 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.647569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.647836 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.647811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.648204 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.648162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.648513 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.648491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-config-out\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.649203 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.649125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.649203 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.649162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.649521 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.649495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.649833 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.649809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-web-config\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:31.653526 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:31.653503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nklh4\" (UniqueName: \"kubernetes.io/projected/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-kube-api-access-nklh4\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:32.148338 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.148294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:32.151000 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.150974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:32.256700 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.256664    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"]
Sep 14 07:50:32.290767 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.290734    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"]
Sep 14 07:50:32.290926 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.290833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.295377 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.295202    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 14 07:50:32.295377 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.295235    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 14 07:50:32.295377 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.295346    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 14 07:50:32.295615 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.295572    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 14 07:50:32.295615 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.295583    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-43ig1v1k2h06s\""
Sep 14 07:50:32.295749 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.295658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 14 07:50:32.296268 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.296249    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7pxp9\""
Sep 14 07:50:32.327906 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.327867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:50:32.452304 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452474 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-grpc-tls\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452474 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452474 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-metrics-client-ca\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452474 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452442    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-tls\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6xhj\" (UniqueName: \"kubernetes.io/projected/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-kube-api-access-b6xhj\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.452660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.452541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553313 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6xhj\" (UniqueName: \"kubernetes.io/projected/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-kube-api-access-b6xhj\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-grpc-tls\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-metrics-client-ca\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.553765 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.553566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-tls\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.555014 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.554932    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-metrics-client-ca\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.556480 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.556429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.556592 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.556505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.556873 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.556829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.557259 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.557236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.557350 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.557321    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-grpc-tls\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.557403 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.557370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-secret-thanos-querier-tls\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.562868 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.562847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6xhj\" (UniqueName: \"kubernetes.io/projected/d5f7c5c6-cc5c-4423-a609-ccfb30f837ac-kube-api-access-b6xhj\") pod \"thanos-querier-b5f4b76bb-ns66k\" (UID: \"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac\") " pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.603915 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:32.603880    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:32.990316 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:32.990269    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad65499_1979_4585_b1da_a643541ee2ac.slice/crio-e58e66ee974f9c47a39907ce1af7fc5f02816815b6bc68026fb5a294a9a66ec4 WatchSource:0}: Error finding container e58e66ee974f9c47a39907ce1af7fc5f02816815b6bc68026fb5a294a9a66ec4: Status 404 returned error can't find the container with id e58e66ee974f9c47a39907ce1af7fc5f02816815b6bc68026fb5a294a9a66ec4
Sep 14 07:50:33.263456 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:33.263358    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2"]
Sep 14 07:50:33.266516 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:33.266315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:50:33.267847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:33.267827    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"]
Sep 14 07:50:33.273476 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:33.273454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8t8p" event={"ID":"5ad65499-1979-4585-b1da-a643541ee2ac","Type":"ContainerStarted","Data":"e58e66ee974f9c47a39907ce1af7fc5f02816815b6bc68026fb5a294a9a66ec4"}
Sep 14 07:50:33.313634 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:33.313600    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b3c120_cc4b_4797_90f8_d6233738cc21.slice/crio-baea82d63df45cbd07b2bc7b8ec6e6f5c1fb37effb06ac1b6399a1b5a6a11481 WatchSource:0}: Error finding container baea82d63df45cbd07b2bc7b8ec6e6f5c1fb37effb06ac1b6399a1b5a6a11481: Status 404 returned error can't find the container with id baea82d63df45cbd07b2bc7b8ec6e6f5c1fb37effb06ac1b6399a1b5a6a11481
Sep 14 07:50:33.314366 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:33.314341    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f5eaa2_1c99_473e_b65e_74a6a2c25f9d.slice/crio-87ca89ce038f9108548fb03f27770632bcfbbf72523e799111a2c1189e98548c WatchSource:0}: Error finding container 87ca89ce038f9108548fb03f27770632bcfbbf72523e799111a2c1189e98548c: Status 404 returned error can't find the container with id 87ca89ce038f9108548fb03f27770632bcfbbf72523e799111a2c1189e98548c
Sep 14 07:50:33.315391 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:33.315373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f7c5c6_cc5c_4423_a609_ccfb30f837ac.slice/crio-b685be36e9a1945c0c0df4e0575851334660c64aaa0a3fc6184a7431c7fcb1bb WatchSource:0}: Error finding container b685be36e9a1945c0c0df4e0575851334660c64aaa0a3fc6184a7431c7fcb1bb: Status 404 returned error can't find the container with id b685be36e9a1945c0c0df4e0575851334660c64aaa0a3fc6184a7431c7fcb1bb
Sep 14 07:50:34.179233 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.179183    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:34.294780 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.290697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8t8p" event={"ID":"5ad65499-1979-4585-b1da-a643541ee2ac","Type":"ContainerStarted","Data":"cfd6753b29509033325c6ac6b1599283d09a3c6f6a9258f4215a30e5865f3953"}
Sep 14 07:50:34.296529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.296379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2" event={"ID":"e2b3c120-cc4b-4797-90f8-d6233738cc21","Type":"ContainerStarted","Data":"ae43edb1ba9fdfce03a83abcccfc9668198a3c17d5f57253b277c5d174151da5"}
Sep 14 07:50:34.296529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.296440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2" event={"ID":"e2b3c120-cc4b-4797-90f8-d6233738cc21","Type":"ContainerStarted","Data":"f6ca8b2be83e7f9608bd730b41d4e9b55a66809236def2ee557527ac238ca050"}
Sep 14 07:50:34.296529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.296474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2" event={"ID":"e2b3c120-cc4b-4797-90f8-d6233738cc21","Type":"ContainerStarted","Data":"baea82d63df45cbd07b2bc7b8ec6e6f5c1fb37effb06ac1b6399a1b5a6a11481"}
Sep 14 07:50:34.301138 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.300217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-fblrm" event={"ID":"9fe2ba75-cb96-46a8-8a04-321885ea1cc7","Type":"ContainerStarted","Data":"9507dfc29eb2018350072adcaecd5c844176f2ede64d7c6e6d227f188d0a15fb"}
Sep 14 07:50:34.301138 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.301076    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:34.304261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.304235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"b685be36e9a1945c0c0df4e0575851334660c64aaa0a3fc6184a7431c7fcb1bb"}
Sep 14 07:50:34.306596 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.306207    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-fblrm"
Sep 14 07:50:34.306596 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.306469    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"87ca89ce038f9108548fb03f27770632bcfbbf72523e799111a2c1189e98548c"}
Sep 14 07:50:34.325336 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:34.325245    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-fblrm" podStartSLOduration=2.438140981 podStartE2EDuration="22.325224828s" totalImagesPullingTime="19.887083847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:34.323316885 +0000 UTC m=+90.031331029" watchObservedRunningTime="2026-09-14 07:50:34.325224828 +0000 UTC m=+90.033238970"
Sep 14 07:50:35.315401 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:35.315359    2563 generic.go:356] "Generic (PLEG): container finished" podID="5ad65499-1979-4585-b1da-a643541ee2ac" containerID="cfd6753b29509033325c6ac6b1599283d09a3c6f6a9258f4215a30e5865f3953" exitCode=0
Sep 14 07:50:35.315946 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:35.315497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8t8p" event={"ID":"5ad65499-1979-4585-b1da-a643541ee2ac","Type":"ContainerDied","Data":"cfd6753b29509033325c6ac6b1599283d09a3c6f6a9258f4215a30e5865f3953"}
Sep 14 07:50:36.449067 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.449020    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:50:36.471473 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.471437    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:50:36.471659 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.471639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.474986 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.474955    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 07:50:36.475272 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.475246    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.475631    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.475703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vknnr\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.475744    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.475644    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.476216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.476263    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.476520    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.476220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.476650    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 07:50:36.476778 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.476668    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 07:50:36.477308 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.477037    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6r8ps4oqhoh24\""
Sep 14 07:50:36.477308 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.477236    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 07:50:36.477308 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.477296    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 07:50:36.477582 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.477559    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 07:50:36.479476 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.479457    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 07:50:36.591818 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.591783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.591958 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.591838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-config-out\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.591958 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.591880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.591958 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.591903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-web-config\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592145 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.591965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592145 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592145 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-config\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592145 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592316 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592161    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592316 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592316 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4xf2\" (UniqueName: \"kubernetes.io/projected/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-kube-api-access-n4xf2\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592316 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592316 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592492    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592541 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.592808 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.592599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693109 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693069    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-config-out\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693315    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-web-config\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693444    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-config\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693498 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4xf2\" (UniqueName: \"kubernetes.io/projected/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-kube-api-access-n4xf2\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693615    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.693867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.694187 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.693953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.694187 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.694039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.694841 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.694564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.697111 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.696765    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.697874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.698437    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.698749    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.698895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.698946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.698949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699679 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.699654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.699793 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.699774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.700176 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.700151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.700800 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.700651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.700917 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.700899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-config-out\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.701277 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.701252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-web-config\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.701564 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.701533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.701667 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.701642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.701866 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.701847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-config\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.707037 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.707018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4xf2\" (UniqueName: \"kubernetes.io/projected/d66a7e8c-1e67-4e42-b388-2cf41495f5cc-kube-api-access-n4xf2\") pod \"prometheus-k8s-0\" (UID: \"d66a7e8c-1e67-4e42-b388-2cf41495f5cc\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:36.785144 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:36.785106    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:37.272502 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.272464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:50:37.278390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:50:37.278332    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66a7e8c_1e67_4e42_b388_2cf41495f5cc.slice/crio-d934c67021bf7281e5a44aaa101d8c0316bf5853d4621bba8729a1a596c71bee WatchSource:0}: Error finding container d934c67021bf7281e5a44aaa101d8c0316bf5853d4621bba8729a1a596c71bee: Status 404 returned error can't find the container with id d934c67021bf7281e5a44aaa101d8c0316bf5853d4621bba8729a1a596c71bee
Sep 14 07:50:37.330772 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.330742    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d" containerID="f0f0ef447efed289ebf75b084ab99de8e1c1f628a13bc48781687bfab68610d0" exitCode=0
Sep 14 07:50:37.330897 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.330863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerDied","Data":"f0f0ef447efed289ebf75b084ab99de8e1c1f628a13bc48781687bfab68610d0"}
Sep 14 07:50:37.334594 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.334572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8t8p" event={"ID":"5ad65499-1979-4585-b1da-a643541ee2ac","Type":"ContainerStarted","Data":"4c16d507bf87c0e77d60ad6dfc174a6c25064feebb155966007c866bf9c83c9e"}
Sep 14 07:50:37.334681 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.334613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8t8p" event={"ID":"5ad65499-1979-4585-b1da-a643541ee2ac","Type":"ContainerStarted","Data":"4e0069381c557db7f963fe2e2efa78eb283248fd66ba4df2b6673aab98684634"}
Sep 14 07:50:37.339904 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.339834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2" event={"ID":"e2b3c120-cc4b-4797-90f8-d6233738cc21","Type":"ContainerStarted","Data":"a890c109e9ececc404fa3b50b851109e94438a2ec052c0ad11290f07d9e125de"}
Sep 14 07:50:37.341957 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.341925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"d934c67021bf7281e5a44aaa101d8c0316bf5853d4621bba8729a1a596c71bee"}
Sep 14 07:50:37.344605 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.344516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"fe6e7507c7dfc07092d85f524cacf363f1e31f1dc7dd9b07fc72cda74afefec5"}
Sep 14 07:50:37.344605 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.344597    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"1032f4115b317eebcba9afa13410ab383d14674daabc20239e5b04d82e48a5c5"}
Sep 14 07:50:37.374354 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.374309    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q8t8p" podStartSLOduration=6.194106091 podStartE2EDuration="7.374292969s" totalImagesPullingTime="1.072490426s" totalInitContainerRuntime="107.696452ms" isStatefulPod=true podCreationTimestamp="2026-09-14 07:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:37.373146543 +0000 UTC m=+93.081160682" watchObservedRunningTime="2026-09-14 07:50:37.374292969 +0000 UTC m=+93.082307111"
Sep 14 07:50:37.394354 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:37.394145    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-djrd2" podStartSLOduration=3.987403508 podStartE2EDuration="7.394125621s" totalImagesPullingTime="3.406722113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:37.392370756 +0000 UTC m=+93.100384899" watchObservedRunningTime="2026-09-14 07:50:37.394125621 +0000 UTC m=+93.102139761"
Sep 14 07:50:38.355239 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:38.355026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerDied","Data":"7e039fb7e37a169b60189b5f14aa8a1298806a2f5aea44427c3c8b68e4761d76"}
Sep 14 07:50:38.356906 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:38.356553    2563 generic.go:356] "Generic (PLEG): container finished" podID="d66a7e8c-1e67-4e42-b388-2cf41495f5cc" containerID="7e039fb7e37a169b60189b5f14aa8a1298806a2f5aea44427c3c8b68e4761d76" exitCode=0
Sep 14 07:50:38.361073 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:38.360979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"eaa77e7d9c7e71f65628ca8d4dc6d6f888ccafffd93925ff1404a7b23c96d771"}
Sep 14 07:50:39.197130 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.197077    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55d4864867-9grlk" podUID="6a0dc390-96dd-4763-889a-d7e8dd8ff645" containerName="registry" containerID="cri-o://c3ab6b5f93b5a784a40e8329223f759599a129b613f0659f6332b523c538ac95" gracePeriod=30
Sep 14 07:50:39.369764 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.369578    2563 generic.go:356] "Generic (PLEG): container finished" podID="6a0dc390-96dd-4763-889a-d7e8dd8ff645" containerID="c3ab6b5f93b5a784a40e8329223f759599a129b613f0659f6332b523c538ac95" exitCode=0
Sep 14 07:50:39.369764 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.369694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d4864867-9grlk" event={"ID":"6a0dc390-96dd-4763-889a-d7e8dd8ff645","Type":"ContainerDied","Data":"c3ab6b5f93b5a784a40e8329223f759599a129b613f0659f6332b523c538ac95"}
Sep 14 07:50:39.553501 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.553284    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729704    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpspx\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-kube-api-access-jpspx\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-trusted-ca\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729812    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-bound-sa-token\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729846    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729880    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-installation-pull-secrets\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-image-registry-private-configuration\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.729958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-certificates\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.730518 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.730012    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a0dc390-96dd-4763-889a-d7e8dd8ff645-ca-trust-extracted\") pod \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\" (UID: \"6a0dc390-96dd-4763-889a-d7e8dd8ff645\") "
Sep 14 07:50:39.731764 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.731714    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:39.732237 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.732183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:39.734532 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.734500    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:39.734709 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.734674    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:39.734957 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.734919    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:39.737077 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.737002    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-kube-api-access-jpspx" (OuterVolumeSpecName: "kube-api-access-jpspx") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "kube-api-access-jpspx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:39.737077 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.737044    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:39.751956 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.751911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0dc390-96dd-4763-889a-d7e8dd8ff645-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6a0dc390-96dd-4763-889a-d7e8dd8ff645" (UID: "6a0dc390-96dd-4763-889a-d7e8dd8ff645"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:50:39.831217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831133    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a0dc390-96dd-4763-889a-d7e8dd8ff645-ca-trust-extracted\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831160    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpspx\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-kube-api-access-jpspx\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831175    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-trusted-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831191    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-bound-sa-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831204    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831216    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-installation-pull-secrets\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831564 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831231    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a0dc390-96dd-4763-889a-d7e8dd8ff645-image-registry-private-configuration\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:39.831564 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:39.831245    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a0dc390-96dd-4763-889a-d7e8dd8ff645-registry-certificates\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:40.381133 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.381054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"51465b3cd494dfb8fc5509802cab736ed1afb1225c4521eb38b2a76ea57c2fe4"}
Sep 14 07:50:40.381529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.381169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"4c887ac777cc501ad5c7924a2126ee41cdba2eabdd846786ffddf37ef2c73d4a"}
Sep 14 07:50:40.381529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.381330    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:40.381529 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.381392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" event={"ID":"d5f7c5c6-cc5c-4423-a609-ccfb30f837ac","Type":"ContainerStarted","Data":"d01d38c7ef77ad1593135136e1a77c6d73b04355b7c426cceb7020b19b04a734"}
Sep 14 07:50:40.385892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.385792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"c09d7756280756955d90abb0c9c7d34e364cff1b50198e74a0ebfb0b7a5b116b"}
Sep 14 07:50:40.385998 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.385936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"9861a34835732eab69036b6d2bcc6ab399bf7907a3cff318a0a8891a7c34ed34"}
Sep 14 07:50:40.385998 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.385980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"f81be2df9bf1f3644a9e10541d41cc23dab9ac8b141e0a27195cbf9a88278e05"}
Sep 14 07:50:40.386159 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.386020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"6d0af65fb49141e9d49d8aac769ce8303c3edec129d9b3a6d890cd084592216d"}
Sep 14 07:50:40.386159 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.386057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"61241a6f3e2d8bf6e696b9c19f43f7070fda9f03a6c2874ed95e7691637c4ae4"}
Sep 14 07:50:40.386159 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.386094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d","Type":"ContainerStarted","Data":"e5eb08402d211a4ab123cdfa49ec96adc87f7beb80c855746d977736ecb947e1"}
Sep 14 07:50:40.387939 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.387915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55d4864867-9grlk" event={"ID":"6a0dc390-96dd-4763-889a-d7e8dd8ff645","Type":"ContainerDied","Data":"724c8864fc4ebb5a07ee1034e9130cf73f6916cd82c5f60da00ad473d2255329"}
Sep 14 07:50:40.388035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.387961    2563 scope.go:117] "RemoveContainer" containerID="c3ab6b5f93b5a784a40e8329223f759599a129b613f0659f6332b523c538ac95"
Sep 14 07:50:40.388035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.387970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55d4864867-9grlk"
Sep 14 07:50:40.405347 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.404244    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k" podStartSLOduration=2.7688578550000003 podStartE2EDuration="8.404226948s" totalImagesPullingTime="5.635369093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:40.402908721 +0000 UTC m=+96.110922862" watchObservedRunningTime="2026-09-14 07:50:40.404226948 +0000 UTC m=+96.112241087"
Sep 14 07:50:40.431408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.431350    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.672370033 podStartE2EDuration="9.431330798s" totalImagesPullingTime="5.718061791s" totalInitContainerRuntime="40.898974ms" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:40.429592595 +0000 UTC m=+96.137606749" watchObservedRunningTime="2026-09-14 07:50:40.431330798 +0000 UTC m=+96.139344938"
Sep 14 07:50:40.444556 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.444526    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55d4864867-9grlk"]
Sep 14 07:50:40.457227 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.457194    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55d4864867-9grlk"]
Sep 14 07:50:40.791836 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:40.791760    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a0dc390-96dd-4763-889a-d7e8dd8ff645" path="/var/lib/kubelet/pods/6a0dc390-96dd-4763-889a-d7e8dd8ff645/volumes"
Sep 14 07:50:43.407481 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:43.407454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"3834d9d1d856552aec746e5b2e30c9ca7fbc70e2808997efde9f79113079966f"}
Sep 14 07:50:43.407782 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:43.407514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"400439b80bd2286180feac23a378a59a5df7565a42fec95aa8aa7d0c6d863d32"}
Sep 14 07:50:43.407782 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:43.407540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"7e0712ef29170bebeb1724415e6514a387c2b36f3ad487b873eb2aff7ae55e30"}
Sep 14 07:50:44.415697 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:44.415664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"2cea9daa8a9c71949596539adc8a0afc40639351111548dbf28def5710103277"}
Sep 14 07:50:44.416052 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:44.415749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"14d9ce3b6233ee851263a311e83a89da3c35092682103d9351f4991e278a1b51"}
Sep 14 07:50:44.416052 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:44.415777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d66a7e8c-1e67-4e42-b388-2cf41495f5cc","Type":"ContainerStarted","Data":"b969700d96e25b59792f95d6ea2a1195d65dc059b0bdb8c819238f47f9985d56"}
Sep 14 07:50:44.444775 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:44.444696    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.775161366 podStartE2EDuration="8.444676283s" totalImagesPullingTime="4.642723355s" totalInitContainerRuntime="26.791562ms" isStatefulPod=false podCreationTimestamp="2026-09-14 07:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:44.443310699 +0000 UTC m=+100.151324838" watchObservedRunningTime="2026-09-14 07:50:44.444676283 +0000 UTC m=+100.152690423"
Sep 14 07:50:45.594507 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:45.594471    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56bcf48cd-px48q"]
Sep 14 07:50:46.401522 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:46.401493    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-b5f4b76bb-ns66k"
Sep 14 07:50:46.788007 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:46.787978    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:50.210853 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:50.210824    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s7jl4"
Sep 14 07:50:55.350035 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.349977    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9f5589f5-scz9n" podUID="32169d13-b2ef-467d-a0a9-049de18fcced" containerName="console" containerID="cri-o://74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f" gracePeriod=15
Sep 14 07:50:55.608829 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.608772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:55.661900 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.661876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-serving-cert\") pod \"32169d13-b2ef-467d-a0a9-049de18fcced\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") "
Sep 14 07:50:55.662045 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.661917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-console-config\") pod \"32169d13-b2ef-467d-a0a9-049de18fcced\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") "
Sep 14 07:50:55.662045 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.661941    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-oauth-serving-cert\") pod \"32169d13-b2ef-467d-a0a9-049de18fcced\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") "
Sep 14 07:50:55.662045 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.661963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8vvn\" (UniqueName: \"kubernetes.io/projected/32169d13-b2ef-467d-a0a9-049de18fcced-kube-api-access-g8vvn\") pod \"32169d13-b2ef-467d-a0a9-049de18fcced\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") "
Sep 14 07:50:55.662206 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-service-ca\") pod \"32169d13-b2ef-467d-a0a9-049de18fcced\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") "
Sep 14 07:50:55.662206 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662153    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-oauth-config\") pod \"32169d13-b2ef-467d-a0a9-049de18fcced\" (UID: \"32169d13-b2ef-467d-a0a9-049de18fcced\") "
Sep 14 07:50:55.662408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662380    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-console-config" (OuterVolumeSpecName: "console-config") pod "32169d13-b2ef-467d-a0a9-049de18fcced" (UID: "32169d13-b2ef-467d-a0a9-049de18fcced"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:55.662487 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662392    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "32169d13-b2ef-467d-a0a9-049de18fcced" (UID: "32169d13-b2ef-467d-a0a9-049de18fcced"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:55.662487 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662467    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-service-ca" (OuterVolumeSpecName: "service-ca") pod "32169d13-b2ef-467d-a0a9-049de18fcced" (UID: "32169d13-b2ef-467d-a0a9-049de18fcced"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:55.662596 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662527    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-console-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:55.662596 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.662541    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-oauth-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:55.664236 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.664210    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "32169d13-b2ef-467d-a0a9-049de18fcced" (UID: "32169d13-b2ef-467d-a0a9-049de18fcced"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:55.664339 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.664301    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "32169d13-b2ef-467d-a0a9-049de18fcced" (UID: "32169d13-b2ef-467d-a0a9-049de18fcced"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:55.664391 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.664345    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32169d13-b2ef-467d-a0a9-049de18fcced-kube-api-access-g8vvn" (OuterVolumeSpecName: "kube-api-access-g8vvn") pod "32169d13-b2ef-467d-a0a9-049de18fcced" (UID: "32169d13-b2ef-467d-a0a9-049de18fcced"). InnerVolumeSpecName "kube-api-access-g8vvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:55.763468 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.763443    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-oauth-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:55.763468 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.763467    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/32169d13-b2ef-467d-a0a9-049de18fcced-console-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:55.763593 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.763481    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8vvn\" (UniqueName: \"kubernetes.io/projected/32169d13-b2ef-467d-a0a9-049de18fcced-kube-api-access-g8vvn\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:55.763593 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:55.763493    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/32169d13-b2ef-467d-a0a9-049de18fcced-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:50:56.464868 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.464832    2563 generic.go:356] "Generic (PLEG): container finished" podID="32169d13-b2ef-467d-a0a9-049de18fcced" containerID="74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f" exitCode=0
Sep 14 07:50:56.465305 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.464877    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f5589f5-scz9n"
Sep 14 07:50:56.465305 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.464889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5589f5-scz9n" event={"ID":"32169d13-b2ef-467d-a0a9-049de18fcced","Type":"ContainerDied","Data":"74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f"}
Sep 14 07:50:56.465305 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.464929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f5589f5-scz9n" event={"ID":"32169d13-b2ef-467d-a0a9-049de18fcced","Type":"ContainerDied","Data":"512aa15a5fa23b0bb6a763d5906a2ee29ad330e3ac6a6f2345619a8559d47dbe"}
Sep 14 07:50:56.465305 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.464949    2563 scope.go:117] "RemoveContainer" containerID="74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f"
Sep 14 07:50:56.474141 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.474125    2563 scope.go:117] "RemoveContainer" containerID="74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f"
Sep 14 07:50:56.474427 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:50:56.474407    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f\": container with ID starting with 74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f not found: ID does not exist" containerID="74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f"
Sep 14 07:50:56.474479 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.474435    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f"} err="failed to get container status \"74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f\": rpc error: code = NotFound desc = could not find container \"74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f\": container with ID starting with 74329f3bca5aff638b846d69c43e21b2193c02027edb4046c0874616b169d71f not found: ID does not exist"
Sep 14 07:50:56.485962 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.485940    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9f5589f5-scz9n"]
Sep 14 07:50:56.489785 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.489755    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9f5589f5-scz9n"]
Sep 14 07:50:56.787459 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:50:56.787387    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32169d13-b2ef-467d-a0a9-049de18fcced" path="/var/lib/kubelet/pods/32169d13-b2ef-467d-a0a9-049de18fcced/volumes"
Sep 14 07:51:10.613869 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.613817    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56bcf48cd-px48q" podUID="b02de419-bff6-44c3-90ab-5d47e38ef82d" containerName="console" containerID="cri-o://2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040" gracePeriod=15
Sep 14 07:51:10.889879 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.889825    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:51:10.985571 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985547    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-oauth-config\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.985719 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-service-ca\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.985719 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985603    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-trusted-ca-bundle\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.985719 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-config\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.985906 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985781    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqpbn\" (UniqueName: \"kubernetes.io/projected/b02de419-bff6-44c3-90ab-5d47e38ef82d-kube-api-access-jqpbn\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.985906 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985838    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-serving-cert\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.985906 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.985882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-oauth-serving-cert\") pod \"b02de419-bff6-44c3-90ab-5d47e38ef82d\" (UID: \"b02de419-bff6-44c3-90ab-5d47e38ef82d\") "
Sep 14 07:51:10.986055 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986038    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-service-ca" (OuterVolumeSpecName: "service-ca") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:10.986113 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986050    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:10.986113 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986064    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-config" (OuterVolumeSpecName: "console-config") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:10.986281 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986229    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:10.986354 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986281    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-trusted-ca-bundle\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:10.986354 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986296    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:10.986434 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.986365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:10.987708 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.987680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:10.987864 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.987840    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:10.987926 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:10.987889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b02de419-bff6-44c3-90ab-5d47e38ef82d-kube-api-access-jqpbn" (OuterVolumeSpecName: "kube-api-access-jqpbn") pod "b02de419-bff6-44c3-90ab-5d47e38ef82d" (UID: "b02de419-bff6-44c3-90ab-5d47e38ef82d"). InnerVolumeSpecName "kube-api-access-jqpbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:11.087445 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.087419    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqpbn\" (UniqueName: \"kubernetes.io/projected/b02de419-bff6-44c3-90ab-5d47e38ef82d-kube-api-access-jqpbn\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:11.087445 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.087442    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:11.087445 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.087451    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b02de419-bff6-44c3-90ab-5d47e38ef82d-oauth-serving-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:11.087618 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.087460    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b02de419-bff6-44c3-90ab-5d47e38ef82d-console-oauth-config\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:51:11.524756 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.524700    2563 generic.go:356] "Generic (PLEG): container finished" podID="b02de419-bff6-44c3-90ab-5d47e38ef82d" containerID="2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040" exitCode=0
Sep 14 07:51:11.524915 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.524770    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bcf48cd-px48q"
Sep 14 07:51:11.524915 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.524792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcf48cd-px48q" event={"ID":"b02de419-bff6-44c3-90ab-5d47e38ef82d","Type":"ContainerDied","Data":"2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040"}
Sep 14 07:51:11.524915 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.524843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bcf48cd-px48q" event={"ID":"b02de419-bff6-44c3-90ab-5d47e38ef82d","Type":"ContainerDied","Data":"2768a6a0c28e8b872ba87b1ef8615df1fc5e19c322b94bf3d9923655cde1960f"}
Sep 14 07:51:11.524915 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.524864    2563 scope.go:117] "RemoveContainer" containerID="2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040"
Sep 14 07:51:11.533690 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.533676    2563 scope.go:117] "RemoveContainer" containerID="2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040"
Sep 14 07:51:11.533961 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:51:11.533944    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040\": container with ID starting with 2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040 not found: ID does not exist" containerID="2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040"
Sep 14 07:51:11.534004 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.533969    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040"} err="failed to get container status \"2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040\": rpc error: code = NotFound desc = could not find container \"2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040\": container with ID starting with 2d908d07d8bf69cd747d21784209e591ec00e8c2afc1360f47ed12921e28f040 not found: ID does not exist"
Sep 14 07:51:11.546462 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.546440    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56bcf48cd-px48q"]
Sep 14 07:51:11.549228 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:11.549207    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56bcf48cd-px48q"]
Sep 14 07:51:12.788209 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:12.788172    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b02de419-bff6-44c3-90ab-5d47e38ef82d" path="/var/lib/kubelet/pods/b02de419-bff6-44c3-90ab-5d47e38ef82d/volumes"
Sep 14 07:51:19.557683 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:19.557635    2563 generic.go:356] "Generic (PLEG): container finished" podID="b8bae1e3-c38a-4e9a-a6a2-e4625a779de4" containerID="ef6818b6487dbfe92604486db895a4e2f3a911a64fe59bd2025378ae160a63d2" exitCode=0
Sep 14 07:51:19.558103 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:19.557719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-8lgjz" event={"ID":"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4","Type":"ContainerDied","Data":"ef6818b6487dbfe92604486db895a4e2f3a911a64fe59bd2025378ae160a63d2"}
Sep 14 07:51:19.558103 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:19.558071    2563 scope.go:117] "RemoveContainer" containerID="ef6818b6487dbfe92604486db895a4e2f3a911a64fe59bd2025378ae160a63d2"
Sep 14 07:51:20.563951 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:20.563920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-8lgjz" event={"ID":"b8bae1e3-c38a-4e9a-a6a2-e4625a779de4","Type":"ContainerStarted","Data":"25531367c9781e3b32f4de1a509bab52c3db83a51be2e3ea5646a33c0ba0290d"}
Sep 14 07:51:36.787766 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:36.787736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:36.805407 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:36.805382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:37.652385 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:37.652359    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:54.560822 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.560784    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8df564599-jc6ng"]
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561121    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b02de419-bff6-44c3-90ab-5d47e38ef82d" containerName="console"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561132    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b02de419-bff6-44c3-90ab-5d47e38ef82d" containerName="console"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561154    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0dc390-96dd-4763-889a-d7e8dd8ff645" containerName="registry"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561159    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0dc390-96dd-4763-889a-d7e8dd8ff645" containerName="registry"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561175    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32169d13-b2ef-467d-a0a9-049de18fcced" containerName="console"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561180    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="32169d13-b2ef-467d-a0a9-049de18fcced" containerName="console"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561226    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0dc390-96dd-4763-889a-d7e8dd8ff645" containerName="registry"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561236    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b02de419-bff6-44c3-90ab-5d47e38ef82d" containerName="console"
Sep 14 07:51:54.561261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.561242    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="32169d13-b2ef-467d-a0a9-049de18fcced" containerName="console"
Sep 14 07:51:54.564783 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.564765    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.568269 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.568251    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 14 07:51:54.569234 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.569217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 14 07:51:54.571031 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.571006    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xtkzk\""
Sep 14 07:51:54.571437 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.571388    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 14 07:51:54.571853 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.571832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 14 07:51:54.574920 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.574898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 14 07:51:54.579018 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.578996    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 14 07:51:54.591739 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.591701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8df564599-jc6ng"]
Sep 14 07:51:54.617978 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.617954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-secret-telemeter-client\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618090 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.617993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-federate-client-tls\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618090 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.618035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618090 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.618075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt72n\" (UniqueName: \"kubernetes.io/projected/48a63f12-7df7-45eb-b81b-cc1126db9bba-kube-api-access-zt72n\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618241 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.618152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-telemeter-client-tls\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618303 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.618236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-metrics-client-ca\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618303 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.618262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.618383 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.618299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-serving-certs-ca-bundle\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.718877 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.718841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-telemeter-client-tls\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.718896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-metrics-client-ca\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.718921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.718948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-serving-certs-ca-bundle\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.718987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-secret-telemeter-client\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.719118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-federate-client-tls\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.719188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.719220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt72n\" (UniqueName: \"kubernetes.io/projected/48a63f12-7df7-45eb-b81b-cc1126db9bba-kube-api-access-zt72n\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.719820 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.719797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-metrics-client-ca\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.720012 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.719987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.720129 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.719811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48a63f12-7df7-45eb-b81b-cc1126db9bba-serving-certs-ca-bundle\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.721578 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.721556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-federate-client-tls\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.721657 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.721586    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-telemeter-client-tls\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.721880 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.721859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.721924 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.721912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/48a63f12-7df7-45eb-b81b-cc1126db9bba-secret-telemeter-client\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.732477 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.732454    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt72n\" (UniqueName: \"kubernetes.io/projected/48a63f12-7df7-45eb-b81b-cc1126db9bba-kube-api-access-zt72n\") pod \"telemeter-client-8df564599-jc6ng\" (UID: \"48a63f12-7df7-45eb-b81b-cc1126db9bba\") " pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:54.873840 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:54.873788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng"
Sep 14 07:51:55.147540 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:55.147418    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8df564599-jc6ng"]
Sep 14 07:51:55.150347 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:51:55.150319    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48a63f12_7df7_45eb_b81b_cc1126db9bba.slice/crio-35d1f77f265e31d493db57a2b902570ad6276017db677626b6f08add30ee64da WatchSource:0}: Error finding container 35d1f77f265e31d493db57a2b902570ad6276017db677626b6f08add30ee64da: Status 404 returned error can't find the container with id 35d1f77f265e31d493db57a2b902570ad6276017db677626b6f08add30ee64da
Sep 14 07:51:55.709925 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:55.709888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng" event={"ID":"48a63f12-7df7-45eb-b81b-cc1126db9bba","Type":"ContainerStarted","Data":"35d1f77f265e31d493db57a2b902570ad6276017db677626b6f08add30ee64da"}
Sep 14 07:51:57.720076 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:57.720041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng" event={"ID":"48a63f12-7df7-45eb-b81b-cc1126db9bba","Type":"ContainerStarted","Data":"99ed70926606ff36c2009981f052f10cf61891bb9b159b633c7f4f0ce19f4e14"}
Sep 14 07:51:57.720417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:57.720104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng" event={"ID":"48a63f12-7df7-45eb-b81b-cc1126db9bba","Type":"ContainerStarted","Data":"5326fb09ad9adaf7b97fc50045839586ed441d509e140a3e94f5f3f8b0f2e942"}
Sep 14 07:51:57.720417 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:57.720124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng" event={"ID":"48a63f12-7df7-45eb-b81b-cc1126db9bba","Type":"ContainerStarted","Data":"3f9230da50d21e7a8b9a4cc1e0e97f47dbc66e38577dd1e8dd6b4c17ee815e05"}
Sep 14 07:51:57.742107 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:51:57.742056    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8df564599-jc6ng" podStartSLOduration=2.206409967 podStartE2EDuration="3.742040667s" totalImagesPullingTime="1.5356307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:51:57.739699264 +0000 UTC m=+173.447713427" watchObservedRunningTime="2026-09-14 07:51:57.742040667 +0000 UTC m=+173.450054806"
Sep 14 07:54:04.660945 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:04.660911    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:54:04.664690 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:04.664670    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:54:04.671720 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:04.671702    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 07:54:47.400129 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.400096    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"]
Sep 14 07:54:47.403207 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.403191    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.405813 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.405786    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 07:54:47.405939 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.405786    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gjdxz\""
Sep 14 07:54:47.406591 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.406574    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:54:47.412814 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.412794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"]
Sep 14 07:54:47.572146 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.572114    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28324170-1ef9-418d-a5a2-6d98c5b387d9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kdcgn\" (UID: \"28324170-1ef9-418d-a5a2-6d98c5b387d9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.572275 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.572163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2gkt\" (UniqueName: \"kubernetes.io/projected/28324170-1ef9-418d-a5a2-6d98c5b387d9-kube-api-access-g2gkt\") pod \"openshift-lws-operator-fd8ccff4c-kdcgn\" (UID: \"28324170-1ef9-418d-a5a2-6d98c5b387d9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.673149 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.673084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28324170-1ef9-418d-a5a2-6d98c5b387d9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kdcgn\" (UID: \"28324170-1ef9-418d-a5a2-6d98c5b387d9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.673149 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.673125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2gkt\" (UniqueName: \"kubernetes.io/projected/28324170-1ef9-418d-a5a2-6d98c5b387d9-kube-api-access-g2gkt\") pod \"openshift-lws-operator-fd8ccff4c-kdcgn\" (UID: \"28324170-1ef9-418d-a5a2-6d98c5b387d9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.673533 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.673513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28324170-1ef9-418d-a5a2-6d98c5b387d9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kdcgn\" (UID: \"28324170-1ef9-418d-a5a2-6d98c5b387d9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.681246 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.681226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2gkt\" (UniqueName: \"kubernetes.io/projected/28324170-1ef9-418d-a5a2-6d98c5b387d9-kube-api-access-g2gkt\") pod \"openshift-lws-operator-fd8ccff4c-kdcgn\" (UID: \"28324170-1ef9-418d-a5a2-6d98c5b387d9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.723138 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.723115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"
Sep 14 07:54:47.993611 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.993568    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn"]
Sep 14 07:54:47.996950 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:54:47.996916    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28324170_1ef9_418d_a5a2_6d98c5b387d9.slice/crio-03b3f523f8e51cdabe72537eabab9d8ac8af0b6f0837c64ddb728d9638b38783 WatchSource:0}: Error finding container 03b3f523f8e51cdabe72537eabab9d8ac8af0b6f0837c64ddb728d9638b38783: Status 404 returned error can't find the container with id 03b3f523f8e51cdabe72537eabab9d8ac8af0b6f0837c64ddb728d9638b38783
Sep 14 07:54:47.998841 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:47.998823    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:54:48.402676 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:48.402599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn" event={"ID":"28324170-1ef9-418d-a5a2-6d98c5b387d9","Type":"ContainerStarted","Data":"03b3f523f8e51cdabe72537eabab9d8ac8af0b6f0837c64ddb728d9638b38783"}
Sep 14 07:54:50.412877 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:50.412840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn" event={"ID":"28324170-1ef9-418d-a5a2-6d98c5b387d9","Type":"ContainerStarted","Data":"9396bf98aef1d3ae0ee1c983c02ead0f66dab36868b654f5a3402d7fcc943975"}
Sep 14 07:54:50.430243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:50.430198    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kdcgn" podStartSLOduration=1.188454025 podStartE2EDuration="3.430184207s" totalImagesPullingTime="2.241730182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:50.428432708 +0000 UTC m=+346.136446857" watchObservedRunningTime="2026-09-14 07:54:50.430184207 +0000 UTC m=+346.138198347"
Sep 14 07:54:57.434745 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.434657    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"]
Sep 14 07:54:57.437572 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.437557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.441770 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.441747    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-92nx9\""
Sep 14 07:54:57.442006 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.441746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 07:54:57.442006 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.441749    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 07:54:57.442173 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.441797    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 07:54:57.448697 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.448673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"]
Sep 14 07:54:57.453099 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.453076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e7762b50-008f-4a2f-855a-bbb9c76fc036-manager-config\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.453187 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.453134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7762b50-008f-4a2f-855a-bbb9c76fc036-cert\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.453238 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.453220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e7762b50-008f-4a2f-855a-bbb9c76fc036-metrics-cert\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.453275 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.453255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n588h\" (UniqueName: \"kubernetes.io/projected/e7762b50-008f-4a2f-855a-bbb9c76fc036-kube-api-access-n588h\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.553637 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.553609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e7762b50-008f-4a2f-855a-bbb9c76fc036-manager-config\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.553781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.553682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7762b50-008f-4a2f-855a-bbb9c76fc036-cert\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.553781 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.553755    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e7762b50-008f-4a2f-855a-bbb9c76fc036-metrics-cert\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.553900 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.553783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n588h\" (UniqueName: \"kubernetes.io/projected/e7762b50-008f-4a2f-855a-bbb9c76fc036-kube-api-access-n588h\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.554261 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.554237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e7762b50-008f-4a2f-855a-bbb9c76fc036-manager-config\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.556189 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.556163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e7762b50-008f-4a2f-855a-bbb9c76fc036-cert\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.556189 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.556178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e7762b50-008f-4a2f-855a-bbb9c76fc036-metrics-cert\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.567318 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.567298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n588h\" (UniqueName: \"kubernetes.io/projected/e7762b50-008f-4a2f-855a-bbb9c76fc036-kube-api-access-n588h\") pod \"lws-controller-manager-7b895dd7c5-mmf9j\" (UID: \"e7762b50-008f-4a2f-855a-bbb9c76fc036\") " pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:57.748694 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:57.748631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:58.018540 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:58.018472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"]
Sep 14 07:54:58.021512 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:54:58.021486    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7762b50_008f_4a2f_855a_bbb9c76fc036.slice/crio-e8272e67cf9fac51135d62cd3d272c2f2e1dfcfe375bf19fb6c4b16af63faf77 WatchSource:0}: Error finding container e8272e67cf9fac51135d62cd3d272c2f2e1dfcfe375bf19fb6c4b16af63faf77: Status 404 returned error can't find the container with id e8272e67cf9fac51135d62cd3d272c2f2e1dfcfe375bf19fb6c4b16af63faf77
Sep 14 07:54:58.448340 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:58.448303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j" event={"ID":"e7762b50-008f-4a2f-855a-bbb9c76fc036","Type":"ContainerStarted","Data":"e8272e67cf9fac51135d62cd3d272c2f2e1dfcfe375bf19fb6c4b16af63faf77"}
Sep 14 07:54:59.454268 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:59.454235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j" event={"ID":"e7762b50-008f-4a2f-855a-bbb9c76fc036","Type":"ContainerStarted","Data":"d34e7915b16de6c3b7d3eb515e297ba0a5ec02da730bf296697d7e0f5b1d31ef"}
Sep 14 07:54:59.454618 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:59.454292    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:54:59.471718 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:54:59.471668    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j" podStartSLOduration=1.203539867 podStartE2EDuration="2.471653165s" totalImagesPullingTime="1.268113298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:54:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:54:59.470512505 +0000 UTC m=+355.178526645" watchObservedRunningTime="2026-09-14 07:54:59.471653165 +0000 UTC m=+355.179667303"
Sep 14 07:55:10.025966 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.025934    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"]
Sep 14 07:55:10.029465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.029448    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.033850 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.033831    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 07:55:10.034030 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.034009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b4wwj\""
Sep 14 07:55:10.044638 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.044620    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 07:55:10.045150 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.045132    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 07:55:10.045554 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.045536    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 07:55:10.081270 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.081247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"]
Sep 14 07:55:10.161506 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.161474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/508738a5-48cb-4f6f-8e5b-3adc538d429f-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.161653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.161526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hlsf\" (UniqueName: \"kubernetes.io/projected/508738a5-48cb-4f6f-8e5b-3adc538d429f-kube-api-access-4hlsf\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.161653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.161566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/508738a5-48cb-4f6f-8e5b-3adc538d429f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.161653 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.161615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/508738a5-48cb-4f6f-8e5b-3adc538d429f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.161798 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.161661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/508738a5-48cb-4f6f-8e5b-3adc538d429f-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.262707 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.262673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/508738a5-48cb-4f6f-8e5b-3adc538d429f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.262892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.262754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/508738a5-48cb-4f6f-8e5b-3adc538d429f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.262892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.262779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/508738a5-48cb-4f6f-8e5b-3adc538d429f-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.262892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.262799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/508738a5-48cb-4f6f-8e5b-3adc538d429f-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.262892 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.262834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hlsf\" (UniqueName: \"kubernetes.io/projected/508738a5-48cb-4f6f-8e5b-3adc538d429f-kube-api-access-4hlsf\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.263286 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.263257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/508738a5-48cb-4f6f-8e5b-3adc538d429f-manifests\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.265056 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.265028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/508738a5-48cb-4f6f-8e5b-3adc538d429f-tmp\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.265164 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.265133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/508738a5-48cb-4f6f-8e5b-3adc538d429f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.265278 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.265257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/508738a5-48cb-4f6f-8e5b-3adc538d429f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.271104 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.271086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hlsf\" (UniqueName: \"kubernetes.io/projected/508738a5-48cb-4f6f-8e5b-3adc538d429f-kube-api-access-4hlsf\") pod \"opendatahub-operator-controller-manager-6c4cb54c55-bhn5g\" (UID: \"508738a5-48cb-4f6f-8e5b-3adc538d429f\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.339122 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.339063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:10.459867 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.459840    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b895dd7c5-mmf9j"
Sep 14 07:55:10.621329 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:10.621253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"]
Sep 14 07:55:10.625390 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:55:10.625362    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508738a5_48cb_4f6f_8e5b_3adc538d429f.slice/crio-fb13b0fa7ae1896e367db72e67a0dc9f3743f29b6fcf66cb5193dac987617ddf WatchSource:0}: Error finding container fb13b0fa7ae1896e367db72e67a0dc9f3743f29b6fcf66cb5193dac987617ddf: Status 404 returned error can't find the container with id fb13b0fa7ae1896e367db72e67a0dc9f3743f29b6fcf66cb5193dac987617ddf
Sep 14 07:55:11.506938 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:11.506903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g" event={"ID":"508738a5-48cb-4f6f-8e5b-3adc538d429f","Type":"ContainerStarted","Data":"fb13b0fa7ae1896e367db72e67a0dc9f3743f29b6fcf66cb5193dac987617ddf"}
Sep 14 07:55:14.523679 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:14.523640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g" event={"ID":"508738a5-48cb-4f6f-8e5b-3adc538d429f","Type":"ContainerStarted","Data":"21a82b2c6b69a75ae5646011817f37a2a665eb26e578dae19a269e67e82f19bc"}
Sep 14 07:55:15.529847 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:15.529806    2563 generic.go:356] "Generic (PLEG): container finished" podID="508738a5-48cb-4f6f-8e5b-3adc538d429f" containerID="21a82b2c6b69a75ae5646011817f37a2a665eb26e578dae19a269e67e82f19bc" exitCode=0
Sep 14 07:55:15.530231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:15.529892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g" event={"ID":"508738a5-48cb-4f6f-8e5b-3adc538d429f","Type":"ContainerDied","Data":"21a82b2c6b69a75ae5646011817f37a2a665eb26e578dae19a269e67e82f19bc"}
Sep 14 07:55:16.536475 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:16.536432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g" event={"ID":"508738a5-48cb-4f6f-8e5b-3adc538d429f","Type":"ContainerStarted","Data":"f40dca6074fea258dcdf64cde973a6435f73711f18320a979c00d799cd5a7a45"}
Sep 14 07:55:16.536855 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:16.536601    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:16.560984 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:16.560934    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g" podStartSLOduration=3.245015351 podStartE2EDuration="7.560919766s" totalImagesPullingTime="3.310331619s" totalInitContainerRuntime="1.005572796s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:16.558709164 +0000 UTC m=+372.266723303" watchObservedRunningTime="2026-09-14 07:55:16.560919766 +0000 UTC m=+372.268933905"
Sep 14 07:55:24.986111 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.986066    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c9f6f8d89-xcgpz"]
Sep 14 07:55:24.994408 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.994382    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:24.996957 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.996935    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 07:55:24.997078 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.996945    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 07:55:24.997078 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.997043    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 07:55:24.998156 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.998134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 07:55:24.998383 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.998231    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hvfj5\""
Sep 14 07:55:24.998616 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:24.998586    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 07:55:25.001383 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.001363    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c9f6f8d89-xcgpz"]
Sep 14 07:55:25.003494 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.003472    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 07:55:25.085232 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-console-config\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.085355 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085236    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-trusted-ca-bundle\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.085355 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1151113c-0ea2-4dcd-9586-ae829213fb09-console-oauth-config\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.085355 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-service-ca\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.085454 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1151113c-0ea2-4dcd-9586-ae829213fb09-console-serving-cert\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.085454 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-oauth-serving-cert\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.085558 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.085452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2vrj\" (UniqueName: \"kubernetes.io/projected/1151113c-0ea2-4dcd-9586-ae829213fb09-kube-api-access-n2vrj\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186199 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-console-config\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186199 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-trusted-ca-bundle\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186359 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1151113c-0ea2-4dcd-9586-ae829213fb09-console-oauth-config\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186403 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-service-ca\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186502 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186475    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1151113c-0ea2-4dcd-9586-ae829213fb09-console-serving-cert\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186615 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-oauth-serving-cert\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.186615 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2vrj\" (UniqueName: \"kubernetes.io/projected/1151113c-0ea2-4dcd-9586-ae829213fb09-kube-api-access-n2vrj\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.187018 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.186986    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-console-config\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.187350 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.187325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-trusted-ca-bundle\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.187350 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.187338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-oauth-serving-cert\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.187467 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.187344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1151113c-0ea2-4dcd-9586-ae829213fb09-service-ca\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.189214 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.189184    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1151113c-0ea2-4dcd-9586-ae829213fb09-console-oauth-config\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.189312 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.189256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1151113c-0ea2-4dcd-9586-ae829213fb09-console-serving-cert\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.194700 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.194678    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2vrj\" (UniqueName: \"kubernetes.io/projected/1151113c-0ea2-4dcd-9586-ae829213fb09-kube-api-access-n2vrj\") pod \"console-6c9f6f8d89-xcgpz\" (UID: \"1151113c-0ea2-4dcd-9586-ae829213fb09\") " pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.306379 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.306319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:25.584780 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:25.584756    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c9f6f8d89-xcgpz"]
Sep 14 07:55:25.587056 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:55:25.587029    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1151113c_0ea2_4dcd_9586_ae829213fb09.slice/crio-a777a1dccb311e3ed7baee7bf9c16efc747ec70ced4c4c8b4b933ac581ef0f91 WatchSource:0}: Error finding container a777a1dccb311e3ed7baee7bf9c16efc747ec70ced4c4c8b4b933ac581ef0f91: Status 404 returned error can't find the container with id a777a1dccb311e3ed7baee7bf9c16efc747ec70ced4c4c8b4b933ac581ef0f91
Sep 14 07:55:26.582029 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:26.581988    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c9f6f8d89-xcgpz" event={"ID":"1151113c-0ea2-4dcd-9586-ae829213fb09","Type":"ContainerStarted","Data":"d9516ba1076422670767939f53608b4516d1b4827fd5e5aaf8a2ed2ed0d845f0"}
Sep 14 07:55:26.582436 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:26.582050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c9f6f8d89-xcgpz" event={"ID":"1151113c-0ea2-4dcd-9586-ae829213fb09","Type":"ContainerStarted","Data":"a777a1dccb311e3ed7baee7bf9c16efc747ec70ced4c4c8b4b933ac581ef0f91"}
Sep 14 07:55:26.599275 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:26.599224    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c9f6f8d89-xcgpz" podStartSLOduration=2.599208589 podStartE2EDuration="2.599208589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:26.598412528 +0000 UTC m=+382.306426673" watchObservedRunningTime="2026-09-14 07:55:26.599208589 +0000 UTC m=+382.307222728"
Sep 14 07:55:27.542716 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:27.542683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4cb54c55-bhn5g"
Sep 14 07:55:30.068117 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.068087    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"]
Sep 14 07:55:30.070398 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.070382    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.072949 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.072921    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 07:55:30.073084 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.072980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 07:55:30.073932 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.073911    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qzx2g\""
Sep 14 07:55:30.074042 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.073954    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 07:55:30.074159 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.074145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 07:55:30.079421 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.079395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"]
Sep 14 07:55:30.229537 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.229499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bqmb\" (UniqueName: \"kubernetes.io/projected/6b2624e5-3340-4603-9c45-f44e3915cb8a-kube-api-access-7bqmb\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.229537 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.229534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b2624e5-3340-4603-9c45-f44e3915cb8a-tls-certs\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.229813 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.229600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b2624e5-3340-4603-9c45-f44e3915cb8a-tmp\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.330432 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.330350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b2624e5-3340-4603-9c45-f44e3915cb8a-tmp\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.330432 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.330423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bqmb\" (UniqueName: \"kubernetes.io/projected/6b2624e5-3340-4603-9c45-f44e3915cb8a-kube-api-access-7bqmb\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.330619 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.330458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b2624e5-3340-4603-9c45-f44e3915cb8a-tls-certs\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.332642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.332618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b2624e5-3340-4603-9c45-f44e3915cb8a-tmp\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.332773 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.332755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b2624e5-3340-4603-9c45-f44e3915cb8a-tls-certs\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.338346 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.338313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bqmb\" (UniqueName: \"kubernetes.io/projected/6b2624e5-3340-4603-9c45-f44e3915cb8a-kube-api-access-7bqmb\") pod \"kube-auth-proxy-75fb8dbff8-kkqw4\" (UID: \"6b2624e5-3340-4603-9c45-f44e3915cb8a\") " pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.381985 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.381960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"
Sep 14 07:55:30.653910 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:30.653877    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4"]
Sep 14 07:55:30.656135 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:55:30.656109    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b2624e5_3340_4603_9c45_f44e3915cb8a.slice/crio-5d40ad5277a4adb46716f9b037280828198d0c469d06dcb21f99b9b43b3dfc4d WatchSource:0}: Error finding container 5d40ad5277a4adb46716f9b037280828198d0c469d06dcb21f99b9b43b3dfc4d: Status 404 returned error can't find the container with id 5d40ad5277a4adb46716f9b037280828198d0c469d06dcb21f99b9b43b3dfc4d
Sep 14 07:55:31.608516 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:31.608466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4" event={"ID":"6b2624e5-3340-4603-9c45-f44e3915cb8a","Type":"ContainerStarted","Data":"5d40ad5277a4adb46716f9b037280828198d0c469d06dcb21f99b9b43b3dfc4d"}
Sep 14 07:55:33.620130 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:33.620039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4" event={"ID":"6b2624e5-3340-4603-9c45-f44e3915cb8a","Type":"ContainerStarted","Data":"b89d519b274bf2b0fd87cc5a9327f89b860728f964070006ceed5ff60d79173a"}
Sep 14 07:55:33.636001 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:33.635952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75fb8dbff8-kkqw4" podStartSLOduration=0.989627491 podStartE2EDuration="3.635934984s" totalImagesPullingTime="2.646307493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:33.635554093 +0000 UTC m=+389.343568224" watchObservedRunningTime="2026-09-14 07:55:33.635934984 +0000 UTC m=+389.343949125"
Sep 14 07:55:35.307545 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:35.307509    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:35.307934 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:35.307597    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:35.312150 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:35.312131    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:55:35.632941 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:55:35.632844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c9f6f8d89-xcgpz"
Sep 14 07:57:12.375167 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.375131    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"]
Sep 14 07:57:12.378610 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.378584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.381601 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.381577    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vb5sh\""
Sep 14 07:57:12.381897 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.381881    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 07:57:12.382075 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.382056    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 07:57:12.404231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.404209    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"]
Sep 14 07:57:12.479154 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.479128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8c7x\" (UniqueName: \"kubernetes.io/projected/c6d988ed-d8dd-4aea-968d-778639d440d6-kube-api-access-f8c7x\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.479302 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.479160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6d988ed-d8dd-4aea-968d-778639d440d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.580636 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.580604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8c7x\" (UniqueName: \"kubernetes.io/projected/c6d988ed-d8dd-4aea-968d-778639d440d6-kube-api-access-f8c7x\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.580783 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.580639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6d988ed-d8dd-4aea-968d-778639d440d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.581016 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.580998    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6d988ed-d8dd-4aea-968d-778639d440d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.589486 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.589466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8c7x\" (UniqueName: \"kubernetes.io/projected/c6d988ed-d8dd-4aea-968d-778639d440d6-kube-api-access-f8c7x\") pod \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.688383 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.688352    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:12.964677 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:12.964657    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"]
Sep 14 07:57:12.967088 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:57:12.967056    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d988ed_d8dd_4aea_968d_778639d440d6.slice/crio-403606e40d230f375a5230366ef630446ca2298cea3ef070f4f74955dfbb532b WatchSource:0}: Error finding container 403606e40d230f375a5230366ef630446ca2298cea3ef070f4f74955dfbb532b: Status 404 returned error can't find the container with id 403606e40d230f375a5230366ef630446ca2298cea3ef070f4f74955dfbb532b
Sep 14 07:57:13.080125 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:13.080099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd" event={"ID":"c6d988ed-d8dd-4aea-968d-778639d440d6","Type":"ContainerStarted","Data":"403606e40d230f375a5230366ef630446ca2298cea3ef070f4f74955dfbb532b"}
Sep 14 07:57:17.104348 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:17.104317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd" event={"ID":"c6d988ed-d8dd-4aea-968d-778639d440d6","Type":"ContainerStarted","Data":"0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb"}
Sep 14 07:57:17.104639 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:17.104387    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:17.132499 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:17.132445    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd" podStartSLOduration=1.084653056 podStartE2EDuration="5.132430817s" totalImagesPullingTime="4.047777761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:17.130960937 +0000 UTC m=+492.838975077" watchObservedRunningTime="2026-09-14 07:57:17.132430817 +0000 UTC m=+492.840444957"
Sep 14 07:57:28.110881 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:28.110848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:30.481193 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.481159    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"]
Sep 14 07:57:30.484565 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.484549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.487025 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.487001    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"]
Sep 14 07:57:30.487197 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.487150    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" containerName="manager" containerID="cri-o://0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb" gracePeriod=2
Sep 14 07:57:30.499896 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.499809    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"]
Sep 14 07:57:30.501750 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.501709    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"]
Sep 14 07:57:30.521623 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:57:30.521594    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:30.536070 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.536041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9vgc\" (UniqueName: \"kubernetes.io/projected/c29ea068-4ce5-469c-8b00-484ecbbe8e53-kube-api-access-s9vgc\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-xh8wx\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.536192 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.536109    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ea068-4ce5-469c-8b00-484ecbbe8e53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-xh8wx\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.637282 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.637209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9vgc\" (UniqueName: \"kubernetes.io/projected/c29ea068-4ce5-469c-8b00-484ecbbe8e53-kube-api-access-s9vgc\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-xh8wx\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.637431 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.637304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ea068-4ce5-469c-8b00-484ecbbe8e53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-xh8wx\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.637805 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.637783    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ea068-4ce5-469c-8b00-484ecbbe8e53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-xh8wx\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.649028 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.649003    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9vgc\" (UniqueName: \"kubernetes.io/projected/c29ea068-4ce5-469c-8b00-484ecbbe8e53-kube-api-access-s9vgc\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-xh8wx\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.732313 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.732261    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:30.734322 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:57:30.734297    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-cgjtd\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:30.737614 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.737595    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8c7x\" (UniqueName: \"kubernetes.io/projected/c6d988ed-d8dd-4aea-968d-778639d440d6-kube-api-access-f8c7x\") pod \"c6d988ed-d8dd-4aea-968d-778639d440d6\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") "
Sep 14 07:57:30.737691 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.737636    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6d988ed-d8dd-4aea-968d-778639d440d6-extensions-socket-volume\") pod \"c6d988ed-d8dd-4aea-968d-778639d440d6\" (UID: \"c6d988ed-d8dd-4aea-968d-778639d440d6\") "
Sep 14 07:57:30.738157 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.738133    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6d988ed-d8dd-4aea-968d-778639d440d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c6d988ed-d8dd-4aea-968d-778639d440d6" (UID: "c6d988ed-d8dd-4aea-968d-778639d440d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:57:30.739642 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.739621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6d988ed-d8dd-4aea-968d-778639d440d6-kube-api-access-f8c7x" (OuterVolumeSpecName: "kube-api-access-f8c7x") pod "c6d988ed-d8dd-4aea-968d-778639d440d6" (UID: "c6d988ed-d8dd-4aea-968d-778639d440d6"). InnerVolumeSpecName "kube-api-access-f8c7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:57:30.790426 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.790400    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" path="/var/lib/kubelet/pods/c6d988ed-d8dd-4aea-968d-778639d440d6/volumes"
Sep 14 07:57:30.796014 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.795996    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:30.838309 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.838266    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8c7x\" (UniqueName: \"kubernetes.io/projected/c6d988ed-d8dd-4aea-968d-778639d440d6-kube-api-access-f8c7x\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:57:30.838309 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:30.838310    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c6d988ed-d8dd-4aea-968d-778639d440d6-extensions-socket-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:57:31.084182 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.084155    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"]
Sep 14 07:57:31.086064 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:57:31.086040    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29ea068_4ce5_469c_8b00_484ecbbe8e53.slice/crio-79eb927a5a7d463b89a4ef76aa1d93d2fa8244fcc7e04e309c47b297b2f36971 WatchSource:0}: Error finding container 79eb927a5a7d463b89a4ef76aa1d93d2fa8244fcc7e04e309c47b297b2f36971: Status 404 returned error can't find the container with id 79eb927a5a7d463b89a4ef76aa1d93d2fa8244fcc7e04e309c47b297b2f36971
Sep 14 07:57:31.170663 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.170637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx" event={"ID":"c29ea068-4ce5-469c-8b00-484ecbbe8e53","Type":"ContainerStarted","Data":"7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1"}
Sep 14 07:57:31.170852 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.170687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx" event={"ID":"c29ea068-4ce5-469c-8b00-484ecbbe8e53","Type":"ContainerStarted","Data":"79eb927a5a7d463b89a4ef76aa1d93d2fa8244fcc7e04e309c47b297b2f36971"}
Sep 14 07:57:31.170852 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.170748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:31.172013 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.171990    2563 generic.go:356] "Generic (PLEG): container finished" podID="c6d988ed-d8dd-4aea-968d-778639d440d6" containerID="0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb" exitCode=0
Sep 14 07:57:31.172088 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.172025    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-cgjtd"
Sep 14 07:57:31.172088 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.172052    2563 scope.go:117] "RemoveContainer" containerID="0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb"
Sep 14 07:57:31.180233 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.180215    2563 scope.go:117] "RemoveContainer" containerID="0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb"
Sep 14 07:57:31.180484 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:57:31.180467    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb\": container with ID starting with 0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb not found: ID does not exist" containerID="0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb"
Sep 14 07:57:31.180538 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.180493    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb"} err="failed to get container status \"0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb\": rpc error: code = NotFound desc = could not find container \"0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb\": container with ID starting with 0ee5c63f7f3d9477c95e7b5ec3edcd6fb32cf05941ec2fccb40485d7fe2ecefb not found: ID does not exist"
Sep 14 07:57:31.198218 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:31.198181    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx" podStartSLOduration=1.198169815 podStartE2EDuration="1.198169815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:31.197465277 +0000 UTC m=+506.905479443" watchObservedRunningTime="2026-09-14 07:57:31.198169815 +0000 UTC m=+506.906183953"
Sep 14 07:57:42.179869 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.179837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:42.484567 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.484535    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"]
Sep 14 07:57:42.484921 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.484908    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" containerName="manager"
Sep 14 07:57:42.484971 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.484923    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" containerName="manager"
Sep 14 07:57:42.485013 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.485003    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6d988ed-d8dd-4aea-968d-778639d440d6" containerName="manager"
Sep 14 07:57:42.489323 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.489305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.501920 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.501898    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"]
Sep 14 07:57:42.533248 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.533221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r56s9\" (UniqueName: \"kubernetes.io/projected/600e5533-3269-418a-b954-a47ab9026920-kube-api-access-r56s9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ksngk\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.533350 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.533276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/600e5533-3269-418a-b954-a47ab9026920-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ksngk\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.634266 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.634236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r56s9\" (UniqueName: \"kubernetes.io/projected/600e5533-3269-418a-b954-a47ab9026920-kube-api-access-r56s9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ksngk\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.634441 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.634292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/600e5533-3269-418a-b954-a47ab9026920-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ksngk\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.634647 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.634630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/600e5533-3269-418a-b954-a47ab9026920-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ksngk\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.651138 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.651110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r56s9\" (UniqueName: \"kubernetes.io/projected/600e5533-3269-418a-b954-a47ab9026920-kube-api-access-r56s9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ksngk\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:42.800234 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:42.800167    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:43.081384 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:43.081309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"]
Sep 14 07:57:43.084697 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:57:43.084672    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600e5533_3269_418a_b954_a47ab9026920.slice/crio-fe88bf8351e9bf63f775867f4fd16a385f60474ce0c393d00b53654324dc885b WatchSource:0}: Error finding container fe88bf8351e9bf63f775867f4fd16a385f60474ce0c393d00b53654324dc885b: Status 404 returned error can't find the container with id fe88bf8351e9bf63f775867f4fd16a385f60474ce0c393d00b53654324dc885b
Sep 14 07:57:43.235362 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:43.235332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk" event={"ID":"600e5533-3269-418a-b954-a47ab9026920","Type":"ContainerStarted","Data":"54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9"}
Sep 14 07:57:43.235762 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:43.235386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk" event={"ID":"600e5533-3269-418a-b954-a47ab9026920","Type":"ContainerStarted","Data":"fe88bf8351e9bf63f775867f4fd16a385f60474ce0c393d00b53654324dc885b"}
Sep 14 07:57:43.235762 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:43.235416    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:43.257414 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:43.257367    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk" podStartSLOduration=1.257349971 podStartE2EDuration="1.257349971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:43.255769277 +0000 UTC m=+518.963783416" watchObservedRunningTime="2026-09-14 07:57:43.257349971 +0000 UTC m=+518.965364114"
Sep 14 07:57:54.243005 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.242975    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 07:57:54.315643 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.315604    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"]
Sep 14 07:57:54.315937 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.315908    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx" podUID="c29ea068-4ce5-469c-8b00-484ecbbe8e53" containerName="manager" containerID="cri-o://7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1" gracePeriod=10
Sep 14 07:57:54.560753 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.560703    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:54.637183 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.637154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9vgc\" (UniqueName: \"kubernetes.io/projected/c29ea068-4ce5-469c-8b00-484ecbbe8e53-kube-api-access-s9vgc\") pod \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") "
Sep 14 07:57:54.637369 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.637201    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ea068-4ce5-469c-8b00-484ecbbe8e53-extensions-socket-volume\") pod \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\" (UID: \"c29ea068-4ce5-469c-8b00-484ecbbe8e53\") "
Sep 14 07:57:54.637592 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.637565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29ea068-4ce5-469c-8b00-484ecbbe8e53-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c29ea068-4ce5-469c-8b00-484ecbbe8e53" (UID: "c29ea068-4ce5-469c-8b00-484ecbbe8e53"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:57:54.639192 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.639173    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29ea068-4ce5-469c-8b00-484ecbbe8e53-kube-api-access-s9vgc" (OuterVolumeSpecName: "kube-api-access-s9vgc") pod "c29ea068-4ce5-469c-8b00-484ecbbe8e53" (UID: "c29ea068-4ce5-469c-8b00-484ecbbe8e53"). InnerVolumeSpecName "kube-api-access-s9vgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:57:54.738660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.738627    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9vgc\" (UniqueName: \"kubernetes.io/projected/c29ea068-4ce5-469c-8b00-484ecbbe8e53-kube-api-access-s9vgc\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:57:54.738660 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:54.738655    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c29ea068-4ce5-469c-8b00-484ecbbe8e53-extensions-socket-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:57:55.297674 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.297624    2563 generic.go:356] "Generic (PLEG): container finished" podID="c29ea068-4ce5-469c-8b00-484ecbbe8e53" containerID="7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1" exitCode=0
Sep 14 07:57:55.298140 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.297786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx" event={"ID":"c29ea068-4ce5-469c-8b00-484ecbbe8e53","Type":"ContainerDied","Data":"7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1"}
Sep 14 07:57:55.298140 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.297871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx" event={"ID":"c29ea068-4ce5-469c-8b00-484ecbbe8e53","Type":"ContainerDied","Data":"79eb927a5a7d463b89a4ef76aa1d93d2fa8244fcc7e04e309c47b297b2f36971"}
Sep 14 07:57:55.298140 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.297911    2563 scope.go:117] "RemoveContainer" containerID="7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1"
Sep 14 07:57:55.298140 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.298101    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"
Sep 14 07:57:55.309951 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.309933    2563 scope.go:117] "RemoveContainer" containerID="7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1"
Sep 14 07:57:55.310244 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:57:55.310220    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1\": container with ID starting with 7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1 not found: ID does not exist" containerID="7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1"
Sep 14 07:57:55.310346 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.310253    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1"} err="failed to get container status \"7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1\": rpc error: code = NotFound desc = could not find container \"7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1\": container with ID starting with 7f68238aee2ebdb42ce719c9d92218282059ed5dcbebbafe2f914af6170a3cb1 not found: ID does not exist"
Sep 14 07:57:55.317925 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.317904    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"]
Sep 14 07:57:55.329391 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:55.329371    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-xh8wx"]
Sep 14 07:57:56.788635 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:57:56.788602    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29ea068-4ce5-469c-8b00-484ecbbe8e53" path="/var/lib/kubelet/pods/c29ea068-4ce5-469c-8b00-484ecbbe8e53/volumes"
Sep 14 07:58:15.174942 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.174911    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-q46rk"]
Sep 14 07:58:15.175322 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.175259    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29ea068-4ce5-469c-8b00-484ecbbe8e53" containerName="manager"
Sep 14 07:58:15.175322 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.175270    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29ea068-4ce5-469c-8b00-484ecbbe8e53" containerName="manager"
Sep 14 07:58:15.175394 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.175339    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29ea068-4ce5-469c-8b00-484ecbbe8e53" containerName="manager"
Sep 14 07:58:15.179808 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.179788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:15.182160 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.182137    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pjmcl\""
Sep 14 07:58:15.187045 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.187022    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-q46rk"]
Sep 14 07:58:15.285832 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.285802    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-b8sd7"]
Sep 14 07:58:15.289107 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.289089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:15.296104 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.296077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b8sd7"]
Sep 14 07:58:15.313833 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.313809    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsbkf\" (UniqueName: \"kubernetes.io/projected/1dafedf6-f721-45f3-90ff-c1a93224caad-kube-api-access-gsbkf\") pod \"authorino-7bcfd8f9cd-q46rk\" (UID: \"1dafedf6-f721-45f3-90ff-c1a93224caad\") " pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:15.414189 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.414164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzv4k\" (UniqueName: \"kubernetes.io/projected/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5-kube-api-access-mzv4k\") pod \"authorino-59c974bd48-b8sd7\" (UID: \"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5\") " pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:15.414331 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.414204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsbkf\" (UniqueName: \"kubernetes.io/projected/1dafedf6-f721-45f3-90ff-c1a93224caad-kube-api-access-gsbkf\") pod \"authorino-7bcfd8f9cd-q46rk\" (UID: \"1dafedf6-f721-45f3-90ff-c1a93224caad\") " pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:15.422626 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.422607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsbkf\" (UniqueName: \"kubernetes.io/projected/1dafedf6-f721-45f3-90ff-c1a93224caad-kube-api-access-gsbkf\") pod \"authorino-7bcfd8f9cd-q46rk\" (UID: \"1dafedf6-f721-45f3-90ff-c1a93224caad\") " pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:15.489503 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.489481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:15.515134 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.515109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzv4k\" (UniqueName: \"kubernetes.io/projected/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5-kube-api-access-mzv4k\") pod \"authorino-59c974bd48-b8sd7\" (UID: \"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5\") " pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:15.522957 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.522928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzv4k\" (UniqueName: \"kubernetes.io/projected/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5-kube-api-access-mzv4k\") pod \"authorino-59c974bd48-b8sd7\" (UID: \"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5\") " pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:15.598815 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.598786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:15.758503 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.758479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-q46rk"]
Sep 14 07:58:15.760210 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:58:15.760183    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dafedf6_f721_45f3_90ff_c1a93224caad.slice/crio-c697e8e95fd1ded48b0d9483007e31bc46eed4668a6686c57ff7ecddb228b275 WatchSource:0}: Error finding container c697e8e95fd1ded48b0d9483007e31bc46eed4668a6686c57ff7ecddb228b275: Status 404 returned error can't find the container with id c697e8e95fd1ded48b0d9483007e31bc46eed4668a6686c57ff7ecddb228b275
Sep 14 07:58:15.867383 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:15.867354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b8sd7"]
Sep 14 07:58:15.870200 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:58:15.870171    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d863b4a_c945_4f57_ac3a_236b6a4a1ad5.slice/crio-60564350c9308e603c41912210c3cb66289a206f68c27293ee064a243ff02ac6 WatchSource:0}: Error finding container 60564350c9308e603c41912210c3cb66289a206f68c27293ee064a243ff02ac6: Status 404 returned error can't find the container with id 60564350c9308e603c41912210c3cb66289a206f68c27293ee064a243ff02ac6
Sep 14 07:58:16.398236 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:16.398202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b8sd7" event={"ID":"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5","Type":"ContainerStarted","Data":"60564350c9308e603c41912210c3cb66289a206f68c27293ee064a243ff02ac6"}
Sep 14 07:58:16.399832 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:16.399809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk" event={"ID":"1dafedf6-f721-45f3-90ff-c1a93224caad","Type":"ContainerStarted","Data":"c697e8e95fd1ded48b0d9483007e31bc46eed4668a6686c57ff7ecddb228b275"}
Sep 14 07:58:18.412465 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:18.412424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk" event={"ID":"1dafedf6-f721-45f3-90ff-c1a93224caad","Type":"ContainerStarted","Data":"624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a"}
Sep 14 07:58:18.413994 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:18.413969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b8sd7" event={"ID":"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5","Type":"ContainerStarted","Data":"21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc"}
Sep 14 07:58:18.426448 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:18.426405    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk" podStartSLOduration=1.432595973 podStartE2EDuration="3.426390458s" totalImagesPullingTime="1.993794485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:18.4253832 +0000 UTC m=+554.133397363" watchObservedRunningTime="2026-09-14 07:58:18.426390458 +0000 UTC m=+554.134404646"
Sep 14 07:58:18.439305 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:18.439258    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-b8sd7" podStartSLOduration=1.550995817 podStartE2EDuration="3.439246651s" totalImagesPullingTime="1.888250834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:18.437236257 +0000 UTC m=+554.145250419" watchObservedRunningTime="2026-09-14 07:58:18.439246651 +0000 UTC m=+554.147260799"
Sep 14 07:58:18.461987 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:18.461959    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-q46rk"]
Sep 14 07:58:20.423202 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:20.423166    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk" podUID="1dafedf6-f721-45f3-90ff-c1a93224caad" containerName="authorino" containerID="cri-o://624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a" gracePeriod=30
Sep 14 07:58:20.671542 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:20.671514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:20.861949 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:20.861919    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsbkf\" (UniqueName: \"kubernetes.io/projected/1dafedf6-f721-45f3-90ff-c1a93224caad-kube-api-access-gsbkf\") pod \"1dafedf6-f721-45f3-90ff-c1a93224caad\" (UID: \"1dafedf6-f721-45f3-90ff-c1a93224caad\") "
Sep 14 07:58:20.863990 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:20.863952    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dafedf6-f721-45f3-90ff-c1a93224caad-kube-api-access-gsbkf" (OuterVolumeSpecName: "kube-api-access-gsbkf") pod "1dafedf6-f721-45f3-90ff-c1a93224caad" (UID: "1dafedf6-f721-45f3-90ff-c1a93224caad"). InnerVolumeSpecName "kube-api-access-gsbkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:58:20.962641 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:20.962611    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsbkf\" (UniqueName: \"kubernetes.io/projected/1dafedf6-f721-45f3-90ff-c1a93224caad-kube-api-access-gsbkf\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:58:21.428674 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.428643    2563 generic.go:356] "Generic (PLEG): container finished" podID="1dafedf6-f721-45f3-90ff-c1a93224caad" containerID="624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a" exitCode=0
Sep 14 07:58:21.429121 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.428683    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk"
Sep 14 07:58:21.429121 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.428710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk" event={"ID":"1dafedf6-f721-45f3-90ff-c1a93224caad","Type":"ContainerDied","Data":"624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a"}
Sep 14 07:58:21.429121 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.428759    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-q46rk" event={"ID":"1dafedf6-f721-45f3-90ff-c1a93224caad","Type":"ContainerDied","Data":"c697e8e95fd1ded48b0d9483007e31bc46eed4668a6686c57ff7ecddb228b275"}
Sep 14 07:58:21.429121 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.428778    2563 scope.go:117] "RemoveContainer" containerID="624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a"
Sep 14 07:58:21.437041 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.437023    2563 scope.go:117] "RemoveContainer" containerID="624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a"
Sep 14 07:58:21.437311 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:58:21.437292    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a\": container with ID starting with 624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a not found: ID does not exist" containerID="624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a"
Sep 14 07:58:21.437351 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.437321    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a"} err="failed to get container status \"624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a\": rpc error: code = NotFound desc = could not find container \"624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a\": container with ID starting with 624cb855ce8710f4e7b6dee7bd9d12bd02c48836f8061eb4d5ea7763b1ecde0a not found: ID does not exist"
Sep 14 07:58:21.448713 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.448690    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-q46rk"]
Sep 14 07:58:21.452339 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:21.452318    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-q46rk"]
Sep 14 07:58:22.788166 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:22.788132    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dafedf6-f721-45f3-90ff-c1a93224caad" path="/var/lib/kubelet/pods/1dafedf6-f721-45f3-90ff-c1a93224caad/volumes"
Sep 14 07:58:43.215020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.214982    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-4bdrs"]
Sep 14 07:58:43.215406 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.215336    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dafedf6-f721-45f3-90ff-c1a93224caad" containerName="authorino"
Sep 14 07:58:43.215406 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.215347    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dafedf6-f721-45f3-90ff-c1a93224caad" containerName="authorino"
Sep 14 07:58:43.215406 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.215403    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dafedf6-f721-45f3-90ff-c1a93224caad" containerName="authorino"
Sep 14 07:58:43.219531 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.219510    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:43.222795 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.222766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-4bdrs"]
Sep 14 07:58:43.243520 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.243495    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-4bdrs"]
Sep 14 07:58:43.243768 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:58:43.243745    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-ffj75], unattached volumes=[], failed to process volumes=[kube-api-access-ffj75]: context canceled" pod="kuadrant-system/authorino-557f876f69-4bdrs" podUID="b528e519-0a52-49e7-9218-baa9f479e0b7"
Sep 14 07:58:43.268439 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.268412    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rcd9k"]
Sep 14 07:58:43.272335 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.272317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.274555 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.274536    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 07:58:43.278004 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.277984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rcd9k"]
Sep 14 07:58:43.332020 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.331997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a1ab98-921a-4ff3-90af-adc2e17984fc-tls-cert\") pod \"authorino-6d4dfc649f-rcd9k\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") " pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.332144 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.332027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcb7v\" (UniqueName: \"kubernetes.io/projected/e8a1ab98-921a-4ff3-90af-adc2e17984fc-kube-api-access-vcb7v\") pod \"authorino-6d4dfc649f-rcd9k\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") " pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.332144 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.332094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffj75\" (UniqueName: \"kubernetes.io/projected/b528e519-0a52-49e7-9218-baa9f479e0b7-kube-api-access-ffj75\") pod \"authorino-557f876f69-4bdrs\" (UID: \"b528e519-0a52-49e7-9218-baa9f479e0b7\") " pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:43.432829 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.432803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a1ab98-921a-4ff3-90af-adc2e17984fc-tls-cert\") pod \"authorino-6d4dfc649f-rcd9k\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") " pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.432925 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.432832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcb7v\" (UniqueName: \"kubernetes.io/projected/e8a1ab98-921a-4ff3-90af-adc2e17984fc-kube-api-access-vcb7v\") pod \"authorino-6d4dfc649f-rcd9k\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") " pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.432925 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.432877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffj75\" (UniqueName: \"kubernetes.io/projected/b528e519-0a52-49e7-9218-baa9f479e0b7-kube-api-access-ffj75\") pod \"authorino-557f876f69-4bdrs\" (UID: \"b528e519-0a52-49e7-9218-baa9f479e0b7\") " pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:43.435276 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.435250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a1ab98-921a-4ff3-90af-adc2e17984fc-tls-cert\") pod \"authorino-6d4dfc649f-rcd9k\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") " pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.440697 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.440672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcb7v\" (UniqueName: \"kubernetes.io/projected/e8a1ab98-921a-4ff3-90af-adc2e17984fc-kube-api-access-vcb7v\") pod \"authorino-6d4dfc649f-rcd9k\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") " pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.440800 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.440672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffj75\" (UniqueName: \"kubernetes.io/projected/b528e519-0a52-49e7-9218-baa9f479e0b7-kube-api-access-ffj75\") pod \"authorino-557f876f69-4bdrs\" (UID: \"b528e519-0a52-49e7-9218-baa9f479e0b7\") " pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:43.527687 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.527611    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rcd9k"]
Sep 14 07:58:43.527949 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.527936    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:43.535360 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.530915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:43.543458 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.543439    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:43.551477 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.551457    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d8d59759f-zhs6j"]
Sep 14 07:58:43.555395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.555379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.561544 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.561522    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d8d59759f-zhs6j"]
Sep 14 07:58:43.636596 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.636570    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffj75\" (UniqueName: \"kubernetes.io/projected/b528e519-0a52-49e7-9218-baa9f479e0b7-kube-api-access-ffj75\") pod \"b528e519-0a52-49e7-9218-baa9f479e0b7\" (UID: \"b528e519-0a52-49e7-9218-baa9f479e0b7\") "
Sep 14 07:58:43.636742 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.636708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58cp\" (UniqueName: \"kubernetes.io/projected/90b67de4-bc86-4d52-99b5-eb41b71f4020-kube-api-access-c58cp\") pod \"authorino-6d8d59759f-zhs6j\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") " pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.636816 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.636782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b67de4-bc86-4d52-99b5-eb41b71f4020-tls-cert\") pod \"authorino-6d8d59759f-zhs6j\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") " pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.638502 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.638474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b528e519-0a52-49e7-9218-baa9f479e0b7-kube-api-access-ffj75" (OuterVolumeSpecName: "kube-api-access-ffj75") pod "b528e519-0a52-49e7-9218-baa9f479e0b7" (UID: "b528e519-0a52-49e7-9218-baa9f479e0b7"). InnerVolumeSpecName "kube-api-access-ffj75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:58:43.737815 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.737788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58cp\" (UniqueName: \"kubernetes.io/projected/90b67de4-bc86-4d52-99b5-eb41b71f4020-kube-api-access-c58cp\") pod \"authorino-6d8d59759f-zhs6j\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") " pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.737920 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.737842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b67de4-bc86-4d52-99b5-eb41b71f4020-tls-cert\") pod \"authorino-6d8d59759f-zhs6j\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") " pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.737920 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.737906    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffj75\" (UniqueName: \"kubernetes.io/projected/b528e519-0a52-49e7-9218-baa9f479e0b7-kube-api-access-ffj75\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:58:43.740103 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.740079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b67de4-bc86-4d52-99b5-eb41b71f4020-tls-cert\") pod \"authorino-6d8d59759f-zhs6j\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") " pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.747841 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.747819    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58cp\" (UniqueName: \"kubernetes.io/projected/90b67de4-bc86-4d52-99b5-eb41b71f4020-kube-api-access-c58cp\") pod \"authorino-6d8d59759f-zhs6j\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") " pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:43.806951 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.806929    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rcd9k"]
Sep 14 07:58:43.809408 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:58:43.809370    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8a1ab98_921a_4ff3_90af_adc2e17984fc.slice/crio-37ac3155eb68e700644611bb7422d0b92429eaf75e03839d391252d3d82496cf WatchSource:0}: Error finding container 37ac3155eb68e700644611bb7422d0b92429eaf75e03839d391252d3d82496cf: Status 404 returned error can't find the container with id 37ac3155eb68e700644611bb7422d0b92429eaf75e03839d391252d3d82496cf
Sep 14 07:58:43.869087 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:43.869062    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 07:58:44.137034 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.137010    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d8d59759f-zhs6j"]
Sep 14 07:58:44.139037 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:58:44.139013    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90b67de4_bc86_4d52_99b5_eb41b71f4020.slice/crio-2ecc06d039f4339010be408f7a11a2d67daf2120665fb02e449c0c5d081abf22 WatchSource:0}: Error finding container 2ecc06d039f4339010be408f7a11a2d67daf2120665fb02e449c0c5d081abf22: Status 404 returned error can't find the container with id 2ecc06d039f4339010be408f7a11a2d67daf2120665fb02e449c0c5d081abf22
Sep 14 07:58:44.536836 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.536798    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8d59759f-zhs6j" event={"ID":"90b67de4-bc86-4d52-99b5-eb41b71f4020","Type":"ContainerStarted","Data":"2ecc06d039f4339010be408f7a11a2d67daf2120665fb02e449c0c5d081abf22"}
Sep 14 07:58:44.538004 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.537978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k" event={"ID":"e8a1ab98-921a-4ff3-90af-adc2e17984fc","Type":"ContainerStarted","Data":"37ac3155eb68e700644611bb7422d0b92429eaf75e03839d391252d3d82496cf"}
Sep 14 07:58:44.538093 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.538019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-4bdrs"
Sep 14 07:58:44.566916 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.566893    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-4bdrs"]
Sep 14 07:58:44.569543 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.569520    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-4bdrs"]
Sep 14 07:58:44.788169 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:44.788096    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b528e519-0a52-49e7-9218-baa9f479e0b7" path="/var/lib/kubelet/pods/b528e519-0a52-49e7-9218-baa9f479e0b7/volumes"
Sep 14 07:58:46.549470 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.549432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8d59759f-zhs6j" event={"ID":"90b67de4-bc86-4d52-99b5-eb41b71f4020","Type":"ContainerStarted","Data":"4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b"}
Sep 14 07:58:46.564359 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.564315    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d8d59759f-zhs6j" podStartSLOduration=2.188130498 podStartE2EDuration="3.564296918s" totalImagesPullingTime="1.37616642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:46.563070427 +0000 UTC m=+582.271084560" watchObservedRunningTime="2026-09-14 07:58:46.564296918 +0000 UTC m=+582.272311056"
Sep 14 07:58:46.592446 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.592416    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b8sd7"]
Sep 14 07:58:46.592627 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.592604    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-b8sd7" podUID="8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" containerName="authorino" containerID="cri-o://21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc" gracePeriod=30
Sep 14 07:58:46.832585 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.832560    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:46.863142 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.863113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzv4k\" (UniqueName: \"kubernetes.io/projected/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5-kube-api-access-mzv4k\") pod \"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5\" (UID: \"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5\") "
Sep 14 07:58:46.865002 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.864979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5-kube-api-access-mzv4k" (OuterVolumeSpecName: "kube-api-access-mzv4k") pod "8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" (UID: "8d863b4a-c945-4f57-ac3a-236b6a4a1ad5"). InnerVolumeSpecName "kube-api-access-mzv4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:58:46.964137 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:46.964114    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzv4k\" (UniqueName: \"kubernetes.io/projected/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5-kube-api-access-mzv4k\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:58:47.555874 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.555830    2563 generic.go:356] "Generic (PLEG): container finished" podID="8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" containerID="21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc" exitCode=0
Sep 14 07:58:47.556326 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.555882    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-b8sd7"
Sep 14 07:58:47.556326 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.555913    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b8sd7" event={"ID":"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5","Type":"ContainerDied","Data":"21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc"}
Sep 14 07:58:47.556326 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.555957    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-b8sd7" event={"ID":"8d863b4a-c945-4f57-ac3a-236b6a4a1ad5","Type":"ContainerDied","Data":"60564350c9308e603c41912210c3cb66289a206f68c27293ee064a243ff02ac6"}
Sep 14 07:58:47.556326 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.555978    2563 scope.go:117] "RemoveContainer" containerID="21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc"
Sep 14 07:58:47.565105 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.565086    2563 scope.go:117] "RemoveContainer" containerID="21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc"
Sep 14 07:58:47.565367 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:58:47.565349    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc\": container with ID starting with 21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc not found: ID does not exist" containerID="21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc"
Sep 14 07:58:47.565406 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.565378    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc"} err="failed to get container status \"21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc\": rpc error: code = NotFound desc = could not find container \"21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc\": container with ID starting with 21ae63928b496e806d5e88cbec98f0e03a1ad1786a96067334117fad4810e0dc not found: ID does not exist"
Sep 14 07:58:47.575494 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.575472    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b8sd7"]
Sep 14 07:58:47.580894 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:47.580874    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-b8sd7"]
Sep 14 07:58:48.788436 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:48.788399    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" path="/var/lib/kubelet/pods/8d863b4a-c945-4f57-ac3a-236b6a4a1ad5/volumes"
Sep 14 07:58:54.591905 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.591862    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k" event={"ID":"e8a1ab98-921a-4ff3-90af-adc2e17984fc","Type":"ContainerStarted","Data":"128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2"}
Sep 14 07:58:54.592318 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.591921    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k" podUID="e8a1ab98-921a-4ff3-90af-adc2e17984fc" containerName="authorino" containerID="cri-o://128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2" gracePeriod=30
Sep 14 07:58:54.607521 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.607480    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k" podStartSLOduration=1.865014841 podStartE2EDuration="11.607464593s" totalImagesPullingTime="9.742449752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:54.605826848 +0000 UTC m=+590.313841058" watchObservedRunningTime="2026-09-14 07:58:54.607464593 +0000 UTC m=+590.315478731"
Sep 14 07:58:54.835894 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.835871    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:54.936731 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.936706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a1ab98-921a-4ff3-90af-adc2e17984fc-tls-cert\") pod \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") "
Sep 14 07:58:54.936860 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.936767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcb7v\" (UniqueName: \"kubernetes.io/projected/e8a1ab98-921a-4ff3-90af-adc2e17984fc-kube-api-access-vcb7v\") pod \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\" (UID: \"e8a1ab98-921a-4ff3-90af-adc2e17984fc\") "
Sep 14 07:58:54.938720 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.938693    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a1ab98-921a-4ff3-90af-adc2e17984fc-kube-api-access-vcb7v" (OuterVolumeSpecName: "kube-api-access-vcb7v") pod "e8a1ab98-921a-4ff3-90af-adc2e17984fc" (UID: "e8a1ab98-921a-4ff3-90af-adc2e17984fc"). InnerVolumeSpecName "kube-api-access-vcb7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:58:54.946830 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:54.946807    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8a1ab98-921a-4ff3-90af-adc2e17984fc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e8a1ab98-921a-4ff3-90af-adc2e17984fc" (UID: "e8a1ab98-921a-4ff3-90af-adc2e17984fc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:58:55.037844 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.037816    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8a1ab98-921a-4ff3-90af-adc2e17984fc-tls-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:58:55.037844 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.037841    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcb7v\" (UniqueName: \"kubernetes.io/projected/e8a1ab98-921a-4ff3-90af-adc2e17984fc-kube-api-access-vcb7v\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 07:58:55.597445 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.597408    2563 generic.go:356] "Generic (PLEG): container finished" podID="e8a1ab98-921a-4ff3-90af-adc2e17984fc" containerID="128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2" exitCode=0
Sep 14 07:58:55.597871 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.597462    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k"
Sep 14 07:58:55.597871 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.597498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k" event={"ID":"e8a1ab98-921a-4ff3-90af-adc2e17984fc","Type":"ContainerDied","Data":"128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2"}
Sep 14 07:58:55.597871 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.597539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d4dfc649f-rcd9k" event={"ID":"e8a1ab98-921a-4ff3-90af-adc2e17984fc","Type":"ContainerDied","Data":"37ac3155eb68e700644611bb7422d0b92429eaf75e03839d391252d3d82496cf"}
Sep 14 07:58:55.597871 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.597556    2563 scope.go:117] "RemoveContainer" containerID="128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2"
Sep 14 07:58:55.614705 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.613384    2563 scope.go:117] "RemoveContainer" containerID="128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2"
Sep 14 07:58:55.614705 ip-10-0-135-172 kubenswrapper[2563]: E0914 07:58:55.613922    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2\": container with ID starting with 128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2 not found: ID does not exist" containerID="128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2"
Sep 14 07:58:55.614705 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.613955    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2"} err="failed to get container status \"128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2\": rpc error: code = NotFound desc = could not find container \"128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2\": container with ID starting with 128095b7a5ece395dedf5b6bd036389b02628d57e070bed4e8093364bc0e47f2 not found: ID does not exist"
Sep 14 07:58:55.621852 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.621830    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rcd9k"]
Sep 14 07:58:55.627550 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:55.627528    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rcd9k"]
Sep 14 07:58:56.789068 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:58:56.789038    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a1ab98-921a-4ff3-90af-adc2e17984fc" path="/var/lib/kubelet/pods/e8a1ab98-921a-4ff3-90af-adc2e17984fc/volumes"
Sep 14 07:59:04.694411 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:04.694384    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:59:04.695001 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:04.694984    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 07:59:44.986057 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986020    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"]
Sep 14 07:59:44.988395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986433    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" containerName="authorino"
Sep 14 07:59:44.988395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986446    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" containerName="authorino"
Sep 14 07:59:44.988395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986456    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a1ab98-921a-4ff3-90af-adc2e17984fc" containerName="authorino"
Sep 14 07:59:44.988395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986461    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a1ab98-921a-4ff3-90af-adc2e17984fc" containerName="authorino"
Sep 14 07:59:44.988395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986519    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d863b4a-c945-4f57-ac3a-236b6a4a1ad5" containerName="authorino"
Sep 14 07:59:44.988395 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.986531    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a1ab98-921a-4ff3-90af-adc2e17984fc" containerName="authorino"
Sep 14 07:59:44.989366 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.989350    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:44.991963 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.991939    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 07:59:44.992904 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.992883    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 07:59:44.993003 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.992938    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 14 07:59:44.993003 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:44.992962    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dsj5g\""
Sep 14 07:59:45.002885 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.002863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"]
Sep 14 07:59:45.040489 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.040460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj8kj\" (UniqueName: \"kubernetes.io/projected/441dbbb3-71f8-4207-bfe3-9497cd26c015-kube-api-access-mj8kj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.040589 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.040497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.040589 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.040541    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.040589 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.040584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441dbbb3-71f8-4207-bfe3-9497cd26c015-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.040688 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.040603    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.040688 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.040646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.141665 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.141631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.141665 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.141666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.141909 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.141696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj8kj\" (UniqueName: \"kubernetes.io/projected/441dbbb3-71f8-4207-bfe3-9497cd26c015-kube-api-access-mj8kj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.141909 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.141737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.141909 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.141781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.141909 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.141839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441dbbb3-71f8-4207-bfe3-9497cd26c015-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.142171 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.142133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.142171 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.142153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.142293 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.142219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.144028 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.144007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/441dbbb3-71f8-4207-bfe3-9497cd26c015-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.144235 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.144220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/441dbbb3-71f8-4207-bfe3-9497cd26c015-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.149026 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.148999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj8kj\" (UniqueName: \"kubernetes.io/projected/441dbbb3-71f8-4207-bfe3-9497cd26c015-kube-api-access-mj8kj\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6\" (UID: \"441dbbb3-71f8-4207-bfe3-9497cd26c015\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.300585 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.300518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:45.573273 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.573247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"]
Sep 14 07:59:45.575607 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:59:45.575544    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441dbbb3_71f8_4207_bfe3_9497cd26c015.slice/crio-cf506009b477d9f12eb9a9d83790b65c1e583efd9eb623f18d88f2c02f7b09f8 WatchSource:0}: Error finding container cf506009b477d9f12eb9a9d83790b65c1e583efd9eb623f18d88f2c02f7b09f8: Status 404 returned error can't find the container with id cf506009b477d9f12eb9a9d83790b65c1e583efd9eb623f18d88f2c02f7b09f8
Sep 14 07:59:45.845287 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:45.845167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6" event={"ID":"441dbbb3-71f8-4207-bfe3-9497cd26c015","Type":"ContainerStarted","Data":"cf506009b477d9f12eb9a9d83790b65c1e583efd9eb623f18d88f2c02f7b09f8"}
Sep 14 07:59:46.851162 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:46.851125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6" event={"ID":"441dbbb3-71f8-4207-bfe3-9497cd26c015","Type":"ContainerStarted","Data":"0d8903164c0711f2192d94de02d7d4f8ef939b3c192c5b77c1ea0eda2a461c7b"}
Sep 14 07:59:46.866356 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:46.865943    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6" podStartSLOduration=1.778403212 podStartE2EDuration="2.865922735s" totalImagesPullingTime="1.087519523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:59:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:46.865214299 +0000 UTC m=+642.573228441" watchObservedRunningTime="2026-09-14 07:59:46.865922735 +0000 UTC m=+642.573936877"
Sep 14 07:59:54.073550 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.073517    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"]
Sep 14 07:59:54.077338 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.077316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.079603 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.079585    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 14 07:59:54.086387 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.086360    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"]
Sep 14 07:59:54.223547 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.223521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.223547 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.223549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.223766 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.223593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.223766 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.223667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5lfm\" (UniqueName: \"kubernetes.io/projected/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-kube-api-access-z5lfm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.223766 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.223736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.223888 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.223772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.324590 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.324505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.324590 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.324553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5lfm\" (UniqueName: \"kubernetes.io/projected/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-kube-api-access-z5lfm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.324837 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.324616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.324837 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.324665    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.324837 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.324701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.324837 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.324750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.325114 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.325094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.325188 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.325118    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.325188 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.325163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.326782 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.326759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.327120 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.327102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.332217 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.332199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5lfm\" (UniqueName: \"kubernetes.io/projected/1cf1d9b0-2a8f-4b12-b444-62abc2a7f897-kube-api-access-z5lfm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn\" (UID: \"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.388821 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.388785    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 07:59:54.658243 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.658217    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"]
Sep 14 07:59:54.660979 ip-10-0-135-172 kubenswrapper[2563]: W0914 07:59:54.660947    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf1d9b0_2a8f_4b12_b444_62abc2a7f897.slice/crio-549a59f0f97948ae90d169980cce21baab208c5e15f568e2fbfe91ac74b1c22a WatchSource:0}: Error finding container 549a59f0f97948ae90d169980cce21baab208c5e15f568e2fbfe91ac74b1c22a: Status 404 returned error can't find the container with id 549a59f0f97948ae90d169980cce21baab208c5e15f568e2fbfe91ac74b1c22a
Sep 14 07:59:54.662385 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.662365    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:59:54.889085 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:54.889055    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn" event={"ID":"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897","Type":"ContainerStarted","Data":"549a59f0f97948ae90d169980cce21baab208c5e15f568e2fbfe91ac74b1c22a"}
Sep 14 07:59:55.301231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:55.301144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:55.301231 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:55.301204    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:55.313881 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:55.313851    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:55.895925 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:55.895885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn" event={"ID":"1cf1d9b0-2a8f-4b12-b444-62abc2a7f897","Type":"ContainerStarted","Data":"93a6bcb5654c64b0b767de275154c911806c3ff332a33acbb5fb6622b3d66467"}
Sep 14 07:59:55.907169 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:55.907144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6"
Sep 14 07:59:55.915431 ip-10-0-135-172 kubenswrapper[2563]: I0914 07:59:55.915393    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn" podStartSLOduration=1.617120714 podStartE2EDuration="1.915378608s" totalImagesPullingTime="298.257894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:55.913633779 +0000 UTC m=+651.621647917" watchObservedRunningTime="2026-09-14 07:59:55.915378608 +0000 UTC m=+651.623393122"
Sep 14 08:00:00.139325 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.139291    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"]
Sep 14 08:00:00.142672 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.142656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"
Sep 14 08:00:00.145081 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.145061    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:00:00.148371 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.148348    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"]
Sep 14 08:00:00.154679 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.154661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"
Sep 14 08:00:00.424665 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.424640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"]
Sep 14 08:00:00.426503 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:00:00.426470    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4c9cb9_69a1_489c_875e_75d6d3e7e63f.slice/crio-dbb882699dc7da3ac5032d3a1561dd392c796140d8da8d3d22045e112986972c WatchSource:0}: Error finding container dbb882699dc7da3ac5032d3a1561dd392c796140d8da8d3d22045e112986972c: Status 404 returned error can't find the container with id dbb882699dc7da3ac5032d3a1561dd392c796140d8da8d3d22045e112986972c
Sep 14 08:00:00.918744 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:00.918693    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v" event={"ID":"7b4c9cb9-69a1-489c-875e-75d6d3e7e63f","Type":"ContainerStarted","Data":"dbb882699dc7da3ac5032d3a1561dd392c796140d8da8d3d22045e112986972c"}
Sep 14 08:00:03.935419 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:03.935376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v" event={"ID":"7b4c9cb9-69a1-489c-875e-75d6d3e7e63f","Type":"ContainerStarted","Data":"f2a8bd1f719b68aa6d2470c6502d9d78cbd090e29aaae5812792b14d29772829"}
Sep 14 08:00:03.949174 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:03.949113    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v" podStartSLOduration=1.272135084 podStartE2EDuration="3.949094068s" totalImagesPullingTime="2.676958984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:03.948711787 +0000 UTC m=+659.656725927" watchObservedRunningTime="2026-09-14 08:00:03.949094068 +0000 UTC m=+659.657108207"
Sep 14 08:00:04.389746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:04.389638    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 08:00:04.389746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:04.389718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 08:00:04.402040 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:04.402018    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 08:00:04.942027 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:04.941989    2563 generic.go:356] "Generic (PLEG): container finished" podID="7b4c9cb9-69a1-489c-875e-75d6d3e7e63f" containerID="f2a8bd1f719b68aa6d2470c6502d9d78cbd090e29aaae5812792b14d29772829" exitCode=0
Sep 14 08:00:04.942387 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:04.942067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v" event={"ID":"7b4c9cb9-69a1-489c-875e-75d6d3e7e63f","Type":"ContainerDied","Data":"f2a8bd1f719b68aa6d2470c6502d9d78cbd090e29aaae5812792b14d29772829"}
Sep 14 08:00:04.953594 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:04.953571    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn"
Sep 14 08:00:06.085401 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:06.085376    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"
Sep 14 08:00:06.953856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:06.953829    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v"
Sep 14 08:00:06.953856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:06.953846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822880-mc46v" event={"ID":"7b4c9cb9-69a1-489c-875e-75d6d3e7e63f","Type":"ContainerDied","Data":"dbb882699dc7da3ac5032d3a1561dd392c796140d8da8d3d22045e112986972c"}
Sep 14 08:00:06.954085 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:06.953891    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbb882699dc7da3ac5032d3a1561dd392c796140d8da8d3d22045e112986972c"
Sep 14 08:00:19.373101 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.373071    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"]
Sep 14 08:00:19.373784 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.373536    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b4c9cb9-69a1-489c-875e-75d6d3e7e63f" containerName="cleanup"
Sep 14 08:00:19.373784 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.373552    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b4c9cb9-69a1-489c-875e-75d6d3e7e63f" containerName="cleanup"
Sep 14 08:00:19.373784 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.373625    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b4c9cb9-69a1-489c-875e-75d6d3e7e63f" containerName="cleanup"
Sep 14 08:00:19.378979 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.378961    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.381470 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.381449    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 14 08:00:19.386510 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.386489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"]
Sep 14 08:00:19.456929 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.456898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.457048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.456936    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.457048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.456969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/62d17039-8bd6-4344-8d20-60affee6097f-kube-api-access-gp8ls\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.457137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.457042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62d17039-8bd6-4344-8d20-60affee6097f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.457137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.457102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.457137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.457129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558150 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558263 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558263 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/62d17039-8bd6-4344-8d20-60affee6097f-kube-api-access-gp8ls\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558263 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62d17039-8bd6-4344-8d20-60affee6097f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558600 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558662 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.558704 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.558677    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.560320 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.560301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62d17039-8bd6-4344-8d20-60affee6097f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.560749 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.560716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62d17039-8bd6-4344-8d20-60affee6097f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.566541 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.566524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp8ls\" (UniqueName: \"kubernetes.io/projected/62d17039-8bd6-4344-8d20-60affee6097f-kube-api-access-gp8ls\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-k6r4w\" (UID: \"62d17039-8bd6-4344-8d20-60affee6097f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.691100 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.691067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:19.961446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:19.961419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"]
Sep 14 08:00:19.963452 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:00:19.963415    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d17039_8bd6_4344_8d20_60affee6097f.slice/crio-ec9171733923493da26db1b47543fd3b7b8c6df4e2033f2370ad7aa2ea385d42 WatchSource:0}: Error finding container ec9171733923493da26db1b47543fd3b7b8c6df4e2033f2370ad7aa2ea385d42: Status 404 returned error can't find the container with id ec9171733923493da26db1b47543fd3b7b8c6df4e2033f2370ad7aa2ea385d42
Sep 14 08:00:20.017562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:20.017527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w" event={"ID":"62d17039-8bd6-4344-8d20-60affee6097f","Type":"ContainerStarted","Data":"ec9171733923493da26db1b47543fd3b7b8c6df4e2033f2370ad7aa2ea385d42"}
Sep 14 08:00:21.024246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:21.024210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w" event={"ID":"62d17039-8bd6-4344-8d20-60affee6097f","Type":"ContainerStarted","Data":"8585060ba768fb8e40f375d80f9bbb60e0dc74455421d4f357b6f6da17a451dd"}
Sep 14 08:00:21.042073 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:21.042014    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w" podStartSLOduration=1.808592055 podStartE2EDuration="2.042000298s" totalImagesPullingTime="233.408243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:21.039969058 +0000 UTC m=+676.747983198" watchObservedRunningTime="2026-09-14 08:00:21.042000298 +0000 UTC m=+676.750014437"
Sep 14 08:00:29.691492 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:29.691461    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:29.691883 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:29.691516    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:29.703594 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:29.703573    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:30.079498 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.079418    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-k6r4w"
Sep 14 08:00:30.883322 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.883285    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"]
Sep 14 08:00:30.886668 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.886648    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:30.889057 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.889035    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 14 08:00:30.894816 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.894796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"]
Sep 14 08:00:30.964507 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.964482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:30.964630 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.964512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:30.964630 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.964567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e18741-812b-4a12-b455-9ae54a4f1ef1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:30.964630 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.964596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl4v2\" (UniqueName: \"kubernetes.io/projected/f4e18741-812b-4a12-b455-9ae54a4f1ef1-kube-api-access-pl4v2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:30.964630 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.964622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:30.964796 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:30.964701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.065570 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.065570 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.065797 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.065797 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e18741-812b-4a12-b455-9ae54a4f1ef1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.065797 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl4v2\" (UniqueName: \"kubernetes.io/projected/f4e18741-812b-4a12-b455-9ae54a4f1ef1-kube-api-access-pl4v2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.065797 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.066012 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.065989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.066091 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.066073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.066154 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.066133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.067871 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.067848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4e18741-812b-4a12-b455-9ae54a4f1ef1-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.068028 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.068009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4e18741-812b-4a12-b455-9ae54a4f1ef1-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.073367 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.073341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl4v2\" (UniqueName: \"kubernetes.io/projected/f4e18741-812b-4a12-b455-9ae54a4f1ef1-kube-api-access-pl4v2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr\" (UID: \"f4e18741-812b-4a12-b455-9ae54a4f1ef1\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.197040 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.197012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:31.468137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:31.468111    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"]
Sep 14 08:00:31.470586 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:00:31.470557    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4e18741_812b_4a12_b455_9ae54a4f1ef1.slice/crio-d03f6b4f6a5774cb12edb9ae336fb80ad99b520bfc04b42d9498e9644e5bc6b4 WatchSource:0}: Error finding container d03f6b4f6a5774cb12edb9ae336fb80ad99b520bfc04b42d9498e9644e5bc6b4: Status 404 returned error can't find the container with id d03f6b4f6a5774cb12edb9ae336fb80ad99b520bfc04b42d9498e9644e5bc6b4
Sep 14 08:00:32.078977 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:32.078939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr" event={"ID":"f4e18741-812b-4a12-b455-9ae54a4f1ef1","Type":"ContainerStarted","Data":"d03f6b4f6a5774cb12edb9ae336fb80ad99b520bfc04b42d9498e9644e5bc6b4"}
Sep 14 08:00:33.085357 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:33.085307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr" event={"ID":"f4e18741-812b-4a12-b455-9ae54a4f1ef1","Type":"ContainerStarted","Data":"d56e3769095d421d62a60fd7ac34fdef72326dd63c725e0b0fffb36545885c3e"}
Sep 14 08:00:33.104703 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:33.104644    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr" podStartSLOduration=1.68593043 podStartE2EDuration="3.104629836s" totalImagesPullingTime="1.418699406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:33.103170269 +0000 UTC m=+688.811184409" watchObservedRunningTime="2026-09-14 08:00:33.104629836 +0000 UTC m=+688.812643985"
Sep 14 08:00:41.197218 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:41.197184    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:41.197573 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:41.197234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:41.209161 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:41.209137    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:00:42.140100 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:00:42.140072    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr"
Sep 14 08:02:29.420228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.420153    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78cb94df66-5sgr9"]
Sep 14 08:02:29.423803 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.423785    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.434187 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.434164    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78cb94df66-5sgr9"]
Sep 14 08:02:29.515404 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.515379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6-kube-api-access-w6srj\") pod \"authorino-78cb94df66-5sgr9\" (UID: \"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6\") " pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.515539 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.515420    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6-tls-cert\") pod \"authorino-78cb94df66-5sgr9\" (UID: \"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6\") " pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.616075 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.616048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6-kube-api-access-w6srj\") pod \"authorino-78cb94df66-5sgr9\" (UID: \"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6\") " pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.616232 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.616107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6-tls-cert\") pod \"authorino-78cb94df66-5sgr9\" (UID: \"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6\") " pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.618568 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.618549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6-tls-cert\") pod \"authorino-78cb94df66-5sgr9\" (UID: \"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6\") " pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.624187 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.624165    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6-kube-api-access-w6srj\") pod \"authorino-78cb94df66-5sgr9\" (UID: \"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6\") " pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:29.732092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:29.732069    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78cb94df66-5sgr9"
Sep 14 08:02:30.006577 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.006544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78cb94df66-5sgr9"]
Sep 14 08:02:30.008496 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:02:30.008462    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d629ada_6ee5_4dc1_a8ee_b8e79e14edb6.slice/crio-b895c30389cb9a721e7c0344948605901c7337dd6dc2b2021f348472de929dc7 WatchSource:0}: Error finding container b895c30389cb9a721e7c0344948605901c7337dd6dc2b2021f348472de929dc7: Status 404 returned error can't find the container with id b895c30389cb9a721e7c0344948605901c7337dd6dc2b2021f348472de929dc7
Sep 14 08:02:30.660905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.660861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78cb94df66-5sgr9" event={"ID":"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6","Type":"ContainerStarted","Data":"66563ccb2ab0a0f706acacf20091adedaa1a684ce773c44ffac2d07779ef2234"}
Sep 14 08:02:30.661351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.660919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78cb94df66-5sgr9" event={"ID":"6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6","Type":"ContainerStarted","Data":"b895c30389cb9a721e7c0344948605901c7337dd6dc2b2021f348472de929dc7"}
Sep 14 08:02:30.697656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.695167    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-78cb94df66-5sgr9" podStartSLOduration=1.160832964 podStartE2EDuration="1.695143273s" totalImagesPullingTime="534.310309ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:30.675359992 +0000 UTC m=+806.383374131" watchObservedRunningTime="2026-09-14 08:02:30.695143273 +0000 UTC m=+806.403157414"
Sep 14 08:02:30.697656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.696475    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8d59759f-zhs6j"]
Sep 14 08:02:30.697656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.696645    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6d8d59759f-zhs6j" podUID="90b67de4-bc86-4d52-99b5-eb41b71f4020" containerName="authorino" containerID="cri-o://4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b" gracePeriod=30
Sep 14 08:02:30.954628 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:30.954600    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 08:02:31.130179 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.130089    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c58cp\" (UniqueName: \"kubernetes.io/projected/90b67de4-bc86-4d52-99b5-eb41b71f4020-kube-api-access-c58cp\") pod \"90b67de4-bc86-4d52-99b5-eb41b71f4020\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") "
Sep 14 08:02:31.130179 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.130136    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b67de4-bc86-4d52-99b5-eb41b71f4020-tls-cert\") pod \"90b67de4-bc86-4d52-99b5-eb41b71f4020\" (UID: \"90b67de4-bc86-4d52-99b5-eb41b71f4020\") "
Sep 14 08:02:31.132217 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.132188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90b67de4-bc86-4d52-99b5-eb41b71f4020-kube-api-access-c58cp" (OuterVolumeSpecName: "kube-api-access-c58cp") pod "90b67de4-bc86-4d52-99b5-eb41b71f4020" (UID: "90b67de4-bc86-4d52-99b5-eb41b71f4020"). InnerVolumeSpecName "kube-api-access-c58cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:31.142036 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.142001    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90b67de4-bc86-4d52-99b5-eb41b71f4020-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "90b67de4-bc86-4d52-99b5-eb41b71f4020" (UID: "90b67de4-bc86-4d52-99b5-eb41b71f4020"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:02:31.231995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.231951    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c58cp\" (UniqueName: \"kubernetes.io/projected/90b67de4-bc86-4d52-99b5-eb41b71f4020-kube-api-access-c58cp\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:02:31.231995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.231983    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90b67de4-bc86-4d52-99b5-eb41b71f4020-tls-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:02:31.668594 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.668559    2563 generic.go:356] "Generic (PLEG): container finished" podID="90b67de4-bc86-4d52-99b5-eb41b71f4020" containerID="4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b" exitCode=0
Sep 14 08:02:31.669005 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.668618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8d59759f-zhs6j" event={"ID":"90b67de4-bc86-4d52-99b5-eb41b71f4020","Type":"ContainerDied","Data":"4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b"}
Sep 14 08:02:31.669005 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.668647    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d8d59759f-zhs6j"
Sep 14 08:02:31.669005 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.668668    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d8d59759f-zhs6j" event={"ID":"90b67de4-bc86-4d52-99b5-eb41b71f4020","Type":"ContainerDied","Data":"2ecc06d039f4339010be408f7a11a2d67daf2120665fb02e449c0c5d081abf22"}
Sep 14 08:02:31.669005 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.668692    2563 scope.go:117] "RemoveContainer" containerID="4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b"
Sep 14 08:02:31.677951 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.677921    2563 scope.go:117] "RemoveContainer" containerID="4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b"
Sep 14 08:02:31.678320 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:02:31.678273    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b\": container with ID starting with 4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b not found: ID does not exist" containerID="4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b"
Sep 14 08:02:31.678405 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.678326    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b"} err="failed to get container status \"4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b\": rpc error: code = NotFound desc = could not find container \"4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b\": container with ID starting with 4145d893ff0e3ebba8c579a350f13afe5ee81344c8cfa0cfa6b89dca78d19a4b not found: ID does not exist"
Sep 14 08:02:31.689514 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.689483    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d8d59759f-zhs6j"]
Sep 14 08:02:31.695365 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:31.695343    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d8d59759f-zhs6j"]
Sep 14 08:02:32.788634 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:32.788599    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90b67de4-bc86-4d52-99b5-eb41b71f4020" path="/var/lib/kubelet/pods/90b67de4-bc86-4d52-99b5-eb41b71f4020/volumes"
Sep 14 08:02:50.169674 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.169640    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7761-ac4f2f"]
Sep 14 08:02:50.170054 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.170042    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90b67de4-bc86-4d52-99b5-eb41b71f4020" containerName="authorino"
Sep 14 08:02:50.170054 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.170055    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="90b67de4-bc86-4d52-99b5-eb41b71f4020" containerName="authorino"
Sep 14 08:02:50.170137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.170127    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="90b67de4-bc86-4d52-99b5-eb41b71f4020" containerName="authorino"
Sep 14 08:02:50.174856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.174833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:02:50.176833 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.176809    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7761-ac4f2f"]
Sep 14 08:02:50.177402 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.177383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qcf9c\""
Sep 14 08:02:50.305264 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.305226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn4xw\" (UniqueName: \"kubernetes.io/projected/107939c5-5677-46a9-b99f-e3f0699373c4-kube-api-access-pn4xw\") pod \"test-curl-7761-ac4f2f\" (UID: \"107939c5-5677-46a9-b99f-e3f0699373c4\") " pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:02:50.406278 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.406244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn4xw\" (UniqueName: \"kubernetes.io/projected/107939c5-5677-46a9-b99f-e3f0699373c4-kube-api-access-pn4xw\") pod \"test-curl-7761-ac4f2f\" (UID: \"107939c5-5677-46a9-b99f-e3f0699373c4\") " pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:02:50.413804 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.413775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn4xw\" (UniqueName: \"kubernetes.io/projected/107939c5-5677-46a9-b99f-e3f0699373c4-kube-api-access-pn4xw\") pod \"test-curl-7761-ac4f2f\" (UID: \"107939c5-5677-46a9-b99f-e3f0699373c4\") " pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:02:50.485427 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.485390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:02:50.753039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.753014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7761-ac4f2f"]
Sep 14 08:02:50.755371 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:02:50.755343    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod107939c5_5677_46a9_b99f_e3f0699373c4.slice/crio-1e110de46a95008f1fb65f43fba3c2762108b6b5a32156740c4e20b68cdcfa2c WatchSource:0}: Error finding container 1e110de46a95008f1fb65f43fba3c2762108b6b5a32156740c4e20b68cdcfa2c: Status 404 returned error can't find the container with id 1e110de46a95008f1fb65f43fba3c2762108b6b5a32156740c4e20b68cdcfa2c
Sep 14 08:02:50.768104 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:50.768077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7761-ac4f2f" event={"ID":"107939c5-5677-46a9-b99f-e3f0699373c4","Type":"ContainerStarted","Data":"1e110de46a95008f1fb65f43fba3c2762108b6b5a32156740c4e20b68cdcfa2c"}
Sep 14 08:02:54.346331 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.346298    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"]
Sep 14 08:02:54.350766 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.350719    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.353226 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.353198    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-cd7582-dockercfg-t5vft\""
Sep 14 08:02:54.353501 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.353481    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-cd7582\""
Sep 14 08:02:54.365918 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.365892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"]
Sep 14 08:02:54.445496 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.445461    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngqcx\" (UniqueName: \"kubernetes.io/projected/2eaa106f-9457-45f8-b699-a2c95aadb1ae-kube-api-access-ngqcx\") pod \"payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.445669 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.445564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eaa106f-9457-45f8-b699-a2c95aadb1ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.546409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.546372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngqcx\" (UniqueName: \"kubernetes.io/projected/2eaa106f-9457-45f8-b699-a2c95aadb1ae-kube-api-access-ngqcx\") pod \"payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.546575 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.546443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eaa106f-9457-45f8-b699-a2c95aadb1ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.547387 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.547359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eaa106f-9457-45f8-b699-a2c95aadb1ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.555208 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.555186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngqcx\" (UniqueName: \"kubernetes.io/projected/2eaa106f-9457-45f8-b699-a2c95aadb1ae-kube-api-access-ngqcx\") pod \"payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.664946 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.664868    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:54.947826 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:54.947798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"]
Sep 14 08:02:55.558381 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:02:55.558352    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eaa106f_9457_45f8_b699_a2c95aadb1ae.slice/crio-4dc671f7b49420306c96eca4e1380edddcea735ba48bfb63e95978e81105039a WatchSource:0}: Error finding container 4dc671f7b49420306c96eca4e1380edddcea735ba48bfb63e95978e81105039a: Status 404 returned error can't find the container with id 4dc671f7b49420306c96eca4e1380edddcea735ba48bfb63e95978e81105039a
Sep 14 08:02:55.583757 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.583719    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"]
Sep 14 08:02:55.586956 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.586941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.589970 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.589950    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-831f8e\""
Sep 14 08:02:55.590534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.590517    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-831f8e-dockercfg-cb7b9\""
Sep 14 08:02:55.598115 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.598094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"]
Sep 14 08:02:55.658324 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.658295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/770de7b7-117d-40db-b82d-462ac10a9986-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.658444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.658351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w48m8\" (UniqueName: \"kubernetes.io/projected/770de7b7-117d-40db-b82d-462ac10a9986-kube-api-access-w48m8\") pod \"payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.759383 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.759349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/770de7b7-117d-40db-b82d-462ac10a9986-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.759502 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.759419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w48m8\" (UniqueName: \"kubernetes.io/projected/770de7b7-117d-40db-b82d-462ac10a9986-kube-api-access-w48m8\") pod \"payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.760033 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.760015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/770de7b7-117d-40db-b82d-462ac10a9986-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.767846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.767828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w48m8\" (UniqueName: \"kubernetes.io/projected/770de7b7-117d-40db-b82d-462ac10a9986-kube-api-access-w48m8\") pod \"payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:55.795355 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.795327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7761-ac4f2f" event={"ID":"107939c5-5677-46a9-b99f-e3f0699373c4","Type":"ContainerStarted","Data":"359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080"}
Sep 14 08:02:55.796589 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.796569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh" event={"ID":"2eaa106f-9457-45f8-b699-a2c95aadb1ae","Type":"ContainerStarted","Data":"4dc671f7b49420306c96eca4e1380edddcea735ba48bfb63e95978e81105039a"}
Sep 14 08:02:55.811225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.811147    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7761-ac4f2f" podStartSLOduration=0.947556431 podStartE2EDuration="5.811135725s" totalImagesPullingTime="4.863579294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:55.809123882 +0000 UTC m=+831.517138023" watchObservedRunningTime="2026-09-14 08:02:55.811135725 +0000 UTC m=+831.519149864"
Sep 14 08:02:55.898374 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:55.898349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:56.172634 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.172608    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"]
Sep 14 08:02:56.174376 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:02:56.174343    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod770de7b7_117d_40db_b82d_462ac10a9986.slice/crio-963098ccc2f0ca539d2d61c093302306994829ac2b4d6eeddf882a87ce07e6f2 WatchSource:0}: Error finding container 963098ccc2f0ca539d2d61c093302306994829ac2b4d6eeddf882a87ce07e6f2: Status 404 returned error can't find the container with id 963098ccc2f0ca539d2d61c093302306994829ac2b4d6eeddf882a87ce07e6f2
Sep 14 08:02:56.410145 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.410110    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7761-ac4f2f"]
Sep 14 08:02:56.414083 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.414055    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7761-ac4f2f"]
Sep 14 08:02:56.531783 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.531750    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7761-788676"]
Sep 14 08:02:56.532294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.532271    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="107939c5-5677-46a9-b99f-e3f0699373c4" containerName="test-curl-7761-ac4f2f"
Sep 14 08:02:56.532294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.532292    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="107939c5-5677-46a9-b99f-e3f0699373c4" containerName="test-curl-7761-ac4f2f"
Sep 14 08:02:56.532458 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.532372    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="107939c5-5677-46a9-b99f-e3f0699373c4" containerName="test-curl-7761-ac4f2f"
Sep 14 08:02:56.535676 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.535656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:02:56.542712 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.542687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7761-788676"]
Sep 14 08:02:56.668074 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.668031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pbmt\" (UniqueName: \"kubernetes.io/projected/4afa1d80-f696-4842-a27c-87275752636c-kube-api-access-8pbmt\") pod \"test-curl-7761-788676\" (UID: \"4afa1d80-f696-4842-a27c-87275752636c\") " pod="opendatahub/test-curl-7761-788676"
Sep 14 08:02:56.769459 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.769424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pbmt\" (UniqueName: \"kubernetes.io/projected/4afa1d80-f696-4842-a27c-87275752636c-kube-api-access-8pbmt\") pod \"test-curl-7761-788676\" (UID: \"4afa1d80-f696-4842-a27c-87275752636c\") " pod="opendatahub/test-curl-7761-788676"
Sep 14 08:02:56.779711 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.779676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pbmt\" (UniqueName: \"kubernetes.io/projected/4afa1d80-f696-4842-a27c-87275752636c-kube-api-access-8pbmt\") pod \"test-curl-7761-788676\" (UID: \"4afa1d80-f696-4842-a27c-87275752636c\") " pod="opendatahub/test-curl-7761-788676"
Sep 14 08:02:56.810475 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.810402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg" event={"ID":"770de7b7-117d-40db-b82d-462ac10a9986","Type":"ContainerStarted","Data":"963098ccc2f0ca539d2d61c093302306994829ac2b4d6eeddf882a87ce07e6f2"}
Sep 14 08:02:56.810614 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:02:56.810500    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7761-ac4f2f" container="test-curl-7761-ac4f2f"
Sep 14 08:02:56.849892 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:56.849864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:02:57.132964 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:57.132938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7761-788676"]
Sep 14 08:02:57.797669 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:02:57.797633    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4afa1d80_f696_4842_a27c_87275752636c.slice/crio-7f59b4a42339337b9ffb639790f6e1c2426a825c4430bfaad485560741e18036 WatchSource:0}: Error finding container 7f59b4a42339337b9ffb639790f6e1c2426a825c4430bfaad485560741e18036: Status 404 returned error can't find the container with id 7f59b4a42339337b9ffb639790f6e1c2426a825c4430bfaad485560741e18036
Sep 14 08:02:57.818620 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:57.818589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7761-788676" event={"ID":"4afa1d80-f696-4842-a27c-87275752636c","Type":"ContainerStarted","Data":"7f59b4a42339337b9ffb639790f6e1c2426a825c4430bfaad485560741e18036"}
Sep 14 08:02:57.818763 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:57.818687    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7761-ac4f2f" podUID="107939c5-5677-46a9-b99f-e3f0699373c4" containerName="test-curl-7761-ac4f2f" containerID="cri-o://359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080" gracePeriod=2
Sep 14 08:02:58.825032 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.824992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7761-788676" event={"ID":"4afa1d80-f696-4842-a27c-87275752636c","Type":"ContainerStarted","Data":"98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273"}
Sep 14 08:02:58.826444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.826413    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg" event={"ID":"770de7b7-117d-40db-b82d-462ac10a9986","Type":"ContainerStarted","Data":"11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6"}
Sep 14 08:02:58.826564 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.826471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:02:58.827775 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.827751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh" event={"ID":"2eaa106f-9457-45f8-b699-a2c95aadb1ae","Type":"ContainerStarted","Data":"2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378"}
Sep 14 08:02:58.827938 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.827921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:02:58.840700 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.840651    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7761-788676" podStartSLOduration=2.840634543 podStartE2EDuration="2.840634543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:58.838050428 +0000 UTC m=+834.546064586" watchObservedRunningTime="2026-09-14 08:02:58.840634543 +0000 UTC m=+834.548648683"
Sep 14 08:02:58.854470 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:58.854423    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg" podStartSLOduration=1.863937458 podStartE2EDuration="3.854406308s" totalImagesPullingTime="1.99046885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:58.853175862 +0000 UTC m=+834.561190002" watchObservedRunningTime="2026-09-14 08:02:58.854406308 +0000 UTC m=+834.562420440"
Sep 14 08:02:59.450590 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:59.450532    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh" podStartSLOduration=2.844506289 podStartE2EDuration="5.450513183s" totalImagesPullingTime="2.606006894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:58.870777585 +0000 UTC m=+834.578791726" watchObservedRunningTime="2026-09-14 08:02:59.450513183 +0000 UTC m=+835.158527321"
Sep 14 08:02:59.451609 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:59.451586    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7761-788676"]
Sep 14 08:02:59.474533 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:02:59.474424    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7761-788676"]
Sep 14 08:02:59.833127 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:02:59.833101    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7761-788676" container="test-curl-7761-788676"
Sep 14 08:03:00.030766 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.030735    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:03:00.101465 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.101392    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn4xw\" (UniqueName: \"kubernetes.io/projected/107939c5-5677-46a9-b99f-e3f0699373c4-kube-api-access-pn4xw\") pod \"107939c5-5677-46a9-b99f-e3f0699373c4\" (UID: \"107939c5-5677-46a9-b99f-e3f0699373c4\") "
Sep 14 08:03:00.103775 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.103742    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/107939c5-5677-46a9-b99f-e3f0699373c4-kube-api-access-pn4xw" (OuterVolumeSpecName: "kube-api-access-pn4xw") pod "107939c5-5677-46a9-b99f-e3f0699373c4" (UID: "107939c5-5677-46a9-b99f-e3f0699373c4"). InnerVolumeSpecName "kube-api-access-pn4xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:00.203110 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.203073    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn4xw\" (UniqueName: \"kubernetes.io/projected/107939c5-5677-46a9-b99f-e3f0699373c4-kube-api-access-pn4xw\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:00.789907 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.789870    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="107939c5-5677-46a9-b99f-e3f0699373c4" path="/var/lib/kubelet/pods/107939c5-5677-46a9-b99f-e3f0699373c4/volumes"
Sep 14 08:03:00.840214 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.840175    2563 generic.go:356] "Generic (PLEG): container finished" podID="107939c5-5677-46a9-b99f-e3f0699373c4" containerID="359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080" exitCode=137
Sep 14 08:03:00.840686 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.840228    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-ac4f2f"
Sep 14 08:03:00.840686 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.840270    2563 scope.go:117] "RemoveContainer" containerID="359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080"
Sep 14 08:03:00.840686 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.840476    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7761-788676" podUID="4afa1d80-f696-4842-a27c-87275752636c" containerName="test-curl-7761-788676" containerID="cri-o://98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273" gracePeriod=2
Sep 14 08:03:00.850577 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.850559    2563 scope.go:117] "RemoveContainer" containerID="359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080"
Sep 14 08:03:00.850933 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:00.850903    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080\": container with ID starting with 359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080 not found: ID does not exist" containerID="359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080"
Sep 14 08:03:00.851054 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:00.850944    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080"} err="failed to get container status \"359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080\": rpc error: code = NotFound desc = could not find container \"359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080\": container with ID starting with 359cbdd912e02ed7f1ddea037bd4ff0847d9ef3872584da2efa714375ac32080 not found: ID does not exist"
Sep 14 08:03:03.029212 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.029184    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:03:03.131140 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.131070    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pbmt\" (UniqueName: \"kubernetes.io/projected/4afa1d80-f696-4842-a27c-87275752636c-kube-api-access-8pbmt\") pod \"4afa1d80-f696-4842-a27c-87275752636c\" (UID: \"4afa1d80-f696-4842-a27c-87275752636c\") "
Sep 14 08:03:03.133011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.132987    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4afa1d80-f696-4842-a27c-87275752636c-kube-api-access-8pbmt" (OuterVolumeSpecName: "kube-api-access-8pbmt") pod "4afa1d80-f696-4842-a27c-87275752636c" (UID: "4afa1d80-f696-4842-a27c-87275752636c"). InnerVolumeSpecName "kube-api-access-8pbmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:03.231911 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.231884    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pbmt\" (UniqueName: \"kubernetes.io/projected/4afa1d80-f696-4842-a27c-87275752636c-kube-api-access-8pbmt\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:03.498557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.498525    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"]
Sep 14 08:03:03.498944 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.498931    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4afa1d80-f696-4842-a27c-87275752636c" containerName="test-curl-7761-788676"
Sep 14 08:03:03.498993 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.498946    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4afa1d80-f696-4842-a27c-87275752636c" containerName="test-curl-7761-788676"
Sep 14 08:03:03.499039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.499030    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4afa1d80-f696-4842-a27c-87275752636c" containerName="test-curl-7761-788676"
Sep 14 08:03:03.502144 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.502124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.504590 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.504561    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9ab35f8f-serving-cert\""
Sep 14 08:03:03.504590 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.504586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:03:03.505845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.505826    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:03:03.505845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.505844    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:03:03.506127 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.505859    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:03:03.509877 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.509859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"]
Sep 14 08:03:03.635416 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.635385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxd26\" (UniqueName: \"kubernetes.io/projected/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-kube-api-access-jxd26\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.635568 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.635426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-metrics-certs\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.635568 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.635494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-openshift-service-ca\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.635568 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.635532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.736510 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.736476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxd26\" (UniqueName: \"kubernetes.io/projected/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-kube-api-access-jxd26\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.736660 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.736525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-metrics-certs\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.736660 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.736584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-openshift-service-ca\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.736660 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.736632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.736892 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:03.736807    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-serving-cert: secret "maas-api-e2e-ait-9ab35f8f-serving-cert" not found
Sep 14 08:03:03.736892 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:03.736891    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls podName:dbf023a5-82bc-4c6e-aba0-8bc10a99b34d nodeName:}" failed. No retries permitted until 2026-09-14 08:03:04.236867754 +0000 UTC m=+839.944881885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls") pod "maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" (UID: "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d") : secret "maas-api-e2e-ait-9ab35f8f-serving-cert" not found
Sep 14 08:03:03.737338 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.737319    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-openshift-service-ca\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.739005 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.738984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-metrics-certs\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.744856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.744831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxd26\" (UniqueName: \"kubernetes.io/projected/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-kube-api-access-jxd26\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:03.858333 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.858252    2563 generic.go:356] "Generic (PLEG): container finished" podID="4afa1d80-f696-4842-a27c-87275752636c" containerID="98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273" exitCode=137
Sep 14 08:03:03.858333 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.858302    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:03:03.858479 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.858356    2563 scope.go:117] "RemoveContainer" containerID="98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273"
Sep 14 08:03:03.867089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.867073    2563 scope.go:117] "RemoveContainer" containerID="98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273"
Sep 14 08:03:03.867364 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:03.867344    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273\": container with ID starting with 98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273 not found: ID does not exist" containerID="98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273"
Sep 14 08:03:03.867420 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:03.867371    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273"} err="failed to get container status \"98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273\": rpc error: code = NotFound desc = could not find container \"98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273\": container with ID starting with 98802bb30f289edb9bc9c77ee44aabee44eece668032ccee0dec6139d2e21273 not found: ID does not exist"
Sep 14 08:03:03.868560 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:03.868535    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7761-788676\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="4afa1d80-f696-4842-a27c-87275752636c" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:03:04.239431 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.239400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:04.242054 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.242028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls\") pod \"maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:04.251909 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.251884    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"]
Sep 14 08:03:04.256578 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.256563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.258694 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:04.258661    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7761-788676\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="4afa1d80-f696-4842-a27c-87275752636c" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:03:04.259168 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.259150    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9ab35f8f-dockercfg-8q9qf\""
Sep 14 08:03:04.259234 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.259170    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9ab35f8f\""
Sep 14 08:03:04.263806 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.263780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"]
Sep 14 08:03:04.282673 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.282647    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"]
Sep 14 08:03:04.285998 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.285982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.287932 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:04.287907    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7761-788676\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="4afa1d80-f696-4842-a27c-87275752636c" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:03:04.293756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.293715    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"]
Sep 14 08:03:04.340510 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.340468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d350e0df-712f-41e1-ae55-c0855b7317a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.340654 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.340550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwlr4\" (UniqueName: \"kubernetes.io/projected/d350e0df-712f-41e1-ae55-c0855b7317a2-kube-api-access-nwlr4\") pod \"payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.416301 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.416274    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:04.441315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.441286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwlr4\" (UniqueName: \"kubernetes.io/projected/d350e0df-712f-41e1-ae55-c0855b7317a2-kube-api-access-nwlr4\") pod \"payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.441436 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.441388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5950856a-3e78-44df-afd6-b0ced841440e-plugins-config-volume\") pod \"payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") " pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.441500 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.441465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d350e0df-712f-41e1-ae55-c0855b7317a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.441553 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.441509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84g4v\" (UniqueName: \"kubernetes.io/projected/5950856a-3e78-44df-afd6-b0ced841440e-kube-api-access-84g4v\") pod \"payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") " pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.442194 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.442171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d350e0df-712f-41e1-ae55-c0855b7317a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.449022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.448988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwlr4\" (UniqueName: \"kubernetes.io/projected/d350e0df-712f-41e1-ae55-c0855b7317a2-kube-api-access-nwlr4\") pod \"payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.542609 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.542533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5950856a-3e78-44df-afd6-b0ced841440e-plugins-config-volume\") pod \"payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") " pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.542609 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.542605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84g4v\" (UniqueName: \"kubernetes.io/projected/5950856a-3e78-44df-afd6-b0ced841440e-kube-api-access-84g4v\") pod \"payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") " pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.543291 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.543268    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5950856a-3e78-44df-afd6-b0ced841440e-plugins-config-volume\") pod \"payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") " pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.550038 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.550013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84g4v\" (UniqueName: \"kubernetes.io/projected/5950856a-3e78-44df-afd6-b0ced841440e-kube-api-access-84g4v\") pod \"payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") " pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.569010 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.568988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:04.597394 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.597369    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:04.702674 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.702639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"]
Sep 14 08:03:04.787602 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:04.787565    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7761-788676\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="4afa1d80-f696-4842-a27c-87275752636c" pod="opendatahub/test-curl-7761-788676"
Sep 14 08:03:04.789387 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.789364    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4afa1d80-f696-4842-a27c-87275752636c" path="/var/lib/kubelet/pods/4afa1d80-f696-4842-a27c-87275752636c/volumes"
Sep 14 08:03:04.845422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.845393    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"]
Sep 14 08:03:04.846756 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:04.846708    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd350e0df_712f_41e1_ae55_c0855b7317a2.slice/crio-6409dc6c9dc882d4080bc141e81d8702a03acc827d3f7e530d4c61276ccf2254 WatchSource:0}: Error finding container 6409dc6c9dc882d4080bc141e81d8702a03acc827d3f7e530d4c61276ccf2254: Status 404 returned error can't find the container with id 6409dc6c9dc882d4080bc141e81d8702a03acc827d3f7e530d4c61276ccf2254
Sep 14 08:03:04.865570 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.865531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz" event={"ID":"d350e0df-712f-41e1-ae55-c0855b7317a2","Type":"ContainerStarted","Data":"6409dc6c9dc882d4080bc141e81d8702a03acc827d3f7e530d4c61276ccf2254"}
Sep 14 08:03:04.867250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.867226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" event={"ID":"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d","Type":"ContainerStarted","Data":"55a9bfb54204a910786e10ff6f1e7f8a24cc9325be6929437b15145d3e3fb007"}
Sep 14 08:03:04.879661 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:04.879638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"]
Sep 14 08:03:04.887912 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:04.887882    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5950856a_3e78_44df_afd6_b0ced841440e.slice/crio-a045fed70f1eb7b53b413cc268dd56d9e2c909019a74a3cbdc6c2fb1c1511bd0 WatchSource:0}: Error finding container a045fed70f1eb7b53b413cc268dd56d9e2c909019a74a3cbdc6c2fb1c1511bd0: Status 404 returned error can't find the container with id a045fed70f1eb7b53b413cc268dd56d9e2c909019a74a3cbdc6c2fb1c1511bd0
Sep 14 08:03:05.875703 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.875665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz" event={"ID":"d350e0df-712f-41e1-ae55-c0855b7317a2","Type":"ContainerStarted","Data":"efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218"}
Sep 14 08:03:05.876177 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.875771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:05.878012 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.877988    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294" event={"ID":"5950856a-3e78-44df-afd6-b0ced841440e","Type":"ContainerStarted","Data":"358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee"}
Sep 14 08:03:05.878121 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.878033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294" event={"ID":"5950856a-3e78-44df-afd6-b0ced841440e","Type":"ContainerStarted","Data":"a045fed70f1eb7b53b413cc268dd56d9e2c909019a74a3cbdc6c2fb1c1511bd0"}
Sep 14 08:03:05.878121 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.878059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:05.894445 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.894394    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz" podStartSLOduration=1.894376354 podStartE2EDuration="1.894376354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:05.890752995 +0000 UTC m=+841.598767134" watchObservedRunningTime="2026-09-14 08:03:05.894376354 +0000 UTC m=+841.602390497"
Sep 14 08:03:05.907383 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:05.907032    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294" podStartSLOduration=1.907016641 podStartE2EDuration="1.907016641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:05.906534973 +0000 UTC m=+841.614549111" watchObservedRunningTime="2026-09-14 08:03:05.907016641 +0000 UTC m=+841.615030783"
Sep 14 08:03:07.891205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.891166    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" event={"ID":"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d","Type":"ContainerStarted","Data":"e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995"}
Sep 14 08:03:07.891840 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.891764    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:07.913434 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.913387    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" podStartSLOduration=2.471376382 podStartE2EDuration="4.91337333s" totalImagesPullingTime="2.441996948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:07.910769027 +0000 UTC m=+843.618783160" watchObservedRunningTime="2026-09-14 08:03:07.91337333 +0000 UTC m=+843.621387469"
Sep 14 08:03:07.924688 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.924663    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"]
Sep 14 08:03:07.928212 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.928197    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:07.931127 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.931108    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-885430\""
Sep 14 08:03:07.945249 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:07.945222    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"]
Sep 14 08:03:08.081554 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.081515    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb70445-936b-4420-be1e-9c15e7767846-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.081755 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.081671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx4bn\" (UniqueName: \"kubernetes.io/projected/9eb70445-936b-4420-be1e-9c15e7767846-kube-api-access-sx4bn\") pod \"payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.182578 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.182546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb70445-936b-4420-be1e-9c15e7767846-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.182759 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.182650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx4bn\" (UniqueName: \"kubernetes.io/projected/9eb70445-936b-4420-be1e-9c15e7767846-kube-api-access-sx4bn\") pod \"payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.183359 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.183338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb70445-936b-4420-be1e-9c15e7767846-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.190377 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.190347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx4bn\" (UniqueName: \"kubernetes.io/projected/9eb70445-936b-4420-be1e-9c15e7767846-kube-api-access-sx4bn\") pod \"payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.238176 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.238139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.517265 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.517236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"]
Sep 14 08:03:08.519935 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:08.519906    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eb70445_936b_4420_be1e_9c15e7767846.slice/crio-5de6ddd1cbc1ae9a84c45f1b229cdb631b8fb67940adffb98f4a635cd2b1ab0f WatchSource:0}: Error finding container 5de6ddd1cbc1ae9a84c45f1b229cdb631b8fb67940adffb98f4a635cd2b1ab0f: Status 404 returned error can't find the container with id 5de6ddd1cbc1ae9a84c45f1b229cdb631b8fb67940adffb98f4a635cd2b1ab0f
Sep 14 08:03:08.897166 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.897088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm" event={"ID":"9eb70445-936b-4420-be1e-9c15e7767846","Type":"ContainerStarted","Data":"21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7"}
Sep 14 08:03:08.897517 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.897248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:08.897517 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.897285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm" event={"ID":"9eb70445-936b-4420-be1e-9c15e7767846","Type":"ContainerStarted","Data":"5de6ddd1cbc1ae9a84c45f1b229cdb631b8fb67940adffb98f4a635cd2b1ab0f"}
Sep 14 08:03:08.913310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:08.913264    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm" podStartSLOduration=1.913250227 podStartE2EDuration="1.913250227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:08.911559183 +0000 UTC m=+844.619573319" watchObservedRunningTime="2026-09-14 08:03:08.913250227 +0000 UTC m=+844.621264365"
Sep 14 08:03:09.835140 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:09.835111    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:03:09.835317 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:09.835175    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:03:13.903155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:13.903128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:16.886100 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:16.886071    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:16.886535 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:16.886156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:19.904641 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:19.904613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:03:20.592957 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.592918    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"]
Sep 14 08:03:20.596626 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.596604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.599077 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.599054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-f67e60-dockercfg-4fjk7\""
Sep 14 08:03:20.599196 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.599071    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-f67e60\""
Sep 14 08:03:20.607033 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.607011    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"]
Sep 14 08:03:20.625555 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.625534    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"]
Sep 14 08:03:20.629000 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.628984    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.635666 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.635638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"]
Sep 14 08:03:20.704292 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.704261    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk9zl\" (UniqueName: \"kubernetes.io/projected/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-kube-api-access-vk9zl\") pod \"payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.704444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.704348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.704444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.704386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvlmp\" (UniqueName: \"kubernetes.io/projected/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-kube-api-access-rvlmp\") pod \"payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.704444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.704411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.805764 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.805711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.805936 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.805773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvlmp\" (UniqueName: \"kubernetes.io/projected/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-kube-api-access-rvlmp\") pod \"payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.805936 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.805792    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.805936 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.805870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk9zl\" (UniqueName: \"kubernetes.io/projected/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-kube-api-access-vk9zl\") pod \"payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.806442 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.806419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.806572 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.806555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.813599 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.813571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvlmp\" (UniqueName: \"kubernetes.io/projected/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-kube-api-access-rvlmp\") pod \"payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:20.813717 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.813699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk9zl\" (UniqueName: \"kubernetes.io/projected/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-kube-api-access-vk9zl\") pod \"payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.908708 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.908632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:20.940762 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:20.940737    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:21.190888 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.190859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"]
Sep 14 08:03:21.193217 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:21.193194    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e950b03_d206_4f81_bc2b_b0f0a61a35d1.slice/crio-502d5b88b9d51d93f6ed239e8cb65c0683aa7e6d4c02f4c78573ae15d98a4667 WatchSource:0}: Error finding container 502d5b88b9d51d93f6ed239e8cb65c0683aa7e6d4c02f4c78573ae15d98a4667: Status 404 returned error can't find the container with id 502d5b88b9d51d93f6ed239e8cb65c0683aa7e6d4c02f4c78573ae15d98a4667
Sep 14 08:03:21.223268 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.223245    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"]
Sep 14 08:03:21.229769 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:21.229741    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dfa45c_65be_431d_a8bf_86d13ecc5ca1.slice/crio-d36ef8394bd5267465a0f3c2da4cda0730bcb79fc9d297f184e7ebf78982f282 WatchSource:0}: Error finding container d36ef8394bd5267465a0f3c2da4cda0730bcb79fc9d297f184e7ebf78982f282: Status 404 returned error can't find the container with id d36ef8394bd5267465a0f3c2da4cda0730bcb79fc9d297f184e7ebf78982f282
Sep 14 08:03:21.341832 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.341792    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"]
Sep 14 08:03:21.346190 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.346168    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:21.348575 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.348553    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:03:21.352974 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.352952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"]
Sep 14 08:03:21.412039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.412010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s\" (UID: \"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:21.512922 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.512847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s\" (UID: \"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:21.513456 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.513439    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s\" (UID: \"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:21.661831 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.661802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:21.942813 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.942781    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"]
Sep 14 08:03:21.947612 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:21.944888    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecbcdd8c_52e6_41ac_a2ee_c60dfd691c67.slice/crio-124d8fa284d6e81c6015989038deabc4d58763ae57a86588f1193f8ea0ae3957 WatchSource:0}: Error finding container 124d8fa284d6e81c6015989038deabc4d58763ae57a86588f1193f8ea0ae3957: Status 404 returned error can't find the container with id 124d8fa284d6e81c6015989038deabc4d58763ae57a86588f1193f8ea0ae3957
Sep 14 08:03:21.972257 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.972216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl" event={"ID":"1e950b03-d206-4f81-bc2b-b0f0a61a35d1","Type":"ContainerStarted","Data":"d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979"}
Sep 14 08:03:21.972374 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.972284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl" event={"ID":"1e950b03-d206-4f81-bc2b-b0f0a61a35d1","Type":"ContainerStarted","Data":"502d5b88b9d51d93f6ed239e8cb65c0683aa7e6d4c02f4c78573ae15d98a4667"}
Sep 14 08:03:21.972480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.972457    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:21.974142 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.974120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc" event={"ID":"08dfa45c-65be-431d-a8bf-86d13ecc5ca1","Type":"ContainerStarted","Data":"e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e"}
Sep 14 08:03:21.974240 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.974171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:21.974240 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.974198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc" event={"ID":"08dfa45c-65be-431d-a8bf-86d13ecc5ca1","Type":"ContainerStarted","Data":"d36ef8394bd5267465a0f3c2da4cda0730bcb79fc9d297f184e7ebf78982f282"}
Sep 14 08:03:21.975533 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:21.975510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s" event={"ID":"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67","Type":"ContainerStarted","Data":"124d8fa284d6e81c6015989038deabc4d58763ae57a86588f1193f8ea0ae3957"}
Sep 14 08:03:22.009950 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:22.009907    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl" podStartSLOduration=2.009889916 podStartE2EDuration="2.009889916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:22.008871187 +0000 UTC m=+857.716885325" watchObservedRunningTime="2026-09-14 08:03:22.009889916 +0000 UTC m=+857.717904054"
Sep 14 08:03:22.052258 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:22.052038    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc" podStartSLOduration=2.052018287 podStartE2EDuration="2.052018287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:22.051128749 +0000 UTC m=+857.759142889" watchObservedRunningTime="2026-09-14 08:03:22.052018287 +0000 UTC m=+857.760032428"
Sep 14 08:03:22.983249 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:22.983215    2563 generic.go:356] "Generic (PLEG): container finished" podID="ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67" containerID="a51ca77d2c90a5494c3aab6ac4f06e2f4d0aab8506dc2fb693cc6a056f219dd8" exitCode=0
Sep 14 08:03:22.983645 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:22.983303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s" event={"ID":"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67","Type":"ContainerDied","Data":"a51ca77d2c90a5494c3aab6ac4f06e2f4d0aab8506dc2fb693cc6a056f219dd8"}
Sep 14 08:03:24.129277 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.129255    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:24.245070 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.244999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67-maas-api-service-ca\") pod \"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67\" (UID: \"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67\") "
Sep 14 08:03:24.245322 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.245300    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67" (UID: "ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:24.346113 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.346084    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:24.994923 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.994897    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"
Sep 14 08:03:24.995086 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.994939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s" event={"ID":"ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67","Type":"ContainerDied","Data":"124d8fa284d6e81c6015989038deabc4d58763ae57a86588f1193f8ea0ae3957"}
Sep 14 08:03:24.995086 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:24.994983    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="124d8fa284d6e81c6015989038deabc4d58763ae57a86588f1193f8ea0ae3957"
Sep 14 08:03:31.973781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:31.973751    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"]
Sep 14 08:03:31.974398 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:31.973951    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" podUID="dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" containerName="maas-api" containerID="cri-o://e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995" gracePeriod=30
Sep 14 08:03:32.040059 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.040025    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"]
Sep 14 08:03:32.040278 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.040252    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294" podUID="5950856a-3e78-44df-afd6-b0ced841440e" containerName="payload-processing" containerID="cri-o://358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee" gracePeriod=30
Sep 14 08:03:32.050753 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.049624    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"]
Sep 14 08:03:32.050753 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.049994    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz" podUID="d350e0df-712f-41e1-ae55-c0855b7317a2" containerName="payload-pre-processing" containerID="cri-o://efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218" gracePeriod=30
Sep 14 08:03:32.188744 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.188695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"]
Sep 14 08:03:32.190695 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.190667    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-9ab35f8f-5bg5s"]
Sep 14 08:03:32.308904 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.308879    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:32.326682 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.326655    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:32.355538 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.355514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:32.427588 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427559    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls\") pod \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") "
Sep 14 08:03:32.427753 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxd26\" (UniqueName: \"kubernetes.io/projected/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-kube-api-access-jxd26\") pod \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") "
Sep 14 08:03:32.427753 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427698    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d350e0df-712f-41e1-ae55-c0855b7317a2-plugins-config-volume\") pod \"d350e0df-712f-41e1-ae55-c0855b7317a2\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") "
Sep 14 08:03:32.427858 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5950856a-3e78-44df-afd6-b0ced841440e-plugins-config-volume\") pod \"5950856a-3e78-44df-afd6-b0ced841440e\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") "
Sep 14 08:03:32.427858 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427804    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwlr4\" (UniqueName: \"kubernetes.io/projected/d350e0df-712f-41e1-ae55-c0855b7317a2-kube-api-access-nwlr4\") pod \"d350e0df-712f-41e1-ae55-c0855b7317a2\" (UID: \"d350e0df-712f-41e1-ae55-c0855b7317a2\") "
Sep 14 08:03:32.427858 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-metrics-certs\") pod \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") "
Sep 14 08:03:32.428021 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-openshift-service-ca\") pod \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\" (UID: \"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d\") "
Sep 14 08:03:32.428021 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.427969    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84g4v\" (UniqueName: \"kubernetes.io/projected/5950856a-3e78-44df-afd6-b0ced841440e-kube-api-access-84g4v\") pod \"5950856a-3e78-44df-afd6-b0ced841440e\" (UID: \"5950856a-3e78-44df-afd6-b0ced841440e\") "
Sep 14 08:03:32.428172 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.428144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d350e0df-712f-41e1-ae55-c0855b7317a2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d350e0df-712f-41e1-ae55-c0855b7317a2" (UID: "d350e0df-712f-41e1-ae55-c0855b7317a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:32.428238 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.428211    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5950856a-3e78-44df-afd6-b0ced841440e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5950856a-3e78-44df-afd6-b0ced841440e" (UID: "5950856a-3e78-44df-afd6-b0ced841440e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:32.428370 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.428351    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d350e0df-712f-41e1-ae55-c0855b7317a2-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.428493 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.428473    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5950856a-3e78-44df-afd6-b0ced841440e-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.428589 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.428561    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" (UID: "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:32.430123 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.430094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" (UID: "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:03:32.430221 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.430116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" (UID: "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:03:32.430221 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.430158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5950856a-3e78-44df-afd6-b0ced841440e-kube-api-access-84g4v" (OuterVolumeSpecName: "kube-api-access-84g4v") pod "5950856a-3e78-44df-afd6-b0ced841440e" (UID: "5950856a-3e78-44df-afd6-b0ced841440e"). InnerVolumeSpecName "kube-api-access-84g4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:32.430305 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.430258    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-kube-api-access-jxd26" (OuterVolumeSpecName: "kube-api-access-jxd26") pod "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" (UID: "dbf023a5-82bc-4c6e-aba0-8bc10a99b34d"). InnerVolumeSpecName "kube-api-access-jxd26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:32.430391 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.430375    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d350e0df-712f-41e1-ae55-c0855b7317a2-kube-api-access-nwlr4" (OuterVolumeSpecName: "kube-api-access-nwlr4") pod "d350e0df-712f-41e1-ae55-c0855b7317a2" (UID: "d350e0df-712f-41e1-ae55-c0855b7317a2"). InnerVolumeSpecName "kube-api-access-nwlr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:32.529285 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.529187    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxd26\" (UniqueName: \"kubernetes.io/projected/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-kube-api-access-jxd26\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.529285 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.529223    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwlr4\" (UniqueName: \"kubernetes.io/projected/d350e0df-712f-41e1-ae55-c0855b7317a2-kube-api-access-nwlr4\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.529285 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.529237    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.529285 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.529250    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.529285 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.529262    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84g4v\" (UniqueName: \"kubernetes.io/projected/5950856a-3e78-44df-afd6-b0ced841440e-kube-api-access-84g4v\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.529285 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.529274    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:03:32.790233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.790162    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67" path="/var/lib/kubelet/pods/ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67/volumes"
Sep 14 08:03:32.985914 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.985887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:03:32.986327 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:32.986107    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:03:33.044176 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.044081    2563 generic.go:356] "Generic (PLEG): container finished" podID="d350e0df-712f-41e1-ae55-c0855b7317a2" containerID="efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218" exitCode=0
Sep 14 08:03:33.044176 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.044141    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"
Sep 14 08:03:33.044406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.044167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz" event={"ID":"d350e0df-712f-41e1-ae55-c0855b7317a2","Type":"ContainerDied","Data":"efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218"}
Sep 14 08:03:33.044406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.044246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz" event={"ID":"d350e0df-712f-41e1-ae55-c0855b7317a2","Type":"ContainerDied","Data":"6409dc6c9dc882d4080bc141e81d8702a03acc827d3f7e530d4c61276ccf2254"}
Sep 14 08:03:33.044406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.044275    2563 scope.go:117] "RemoveContainer" containerID="efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218"
Sep 14 08:03:33.047253 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.047226    2563 generic.go:356] "Generic (PLEG): container finished" podID="dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" containerID="e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995" exitCode=0
Sep 14 08:03:33.047409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.047362    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" event={"ID":"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d","Type":"ContainerDied","Data":"e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995"}
Sep 14 08:03:33.047409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.047402    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"
Sep 14 08:03:33.047892 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.047403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg" event={"ID":"dbf023a5-82bc-4c6e-aba0-8bc10a99b34d","Type":"ContainerDied","Data":"55a9bfb54204a910786e10ff6f1e7f8a24cc9325be6929437b15145d3e3fb007"}
Sep 14 08:03:33.050086 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.050064    2563 generic.go:356] "Generic (PLEG): container finished" podID="5950856a-3e78-44df-afd6-b0ced841440e" containerID="358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee" exitCode=0
Sep 14 08:03:33.050236 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.050116    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"
Sep 14 08:03:33.050236 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.050134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294" event={"ID":"5950856a-3e78-44df-afd6-b0ced841440e","Type":"ContainerDied","Data":"358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee"}
Sep 14 08:03:33.050236 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.050173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294" event={"ID":"5950856a-3e78-44df-afd6-b0ced841440e","Type":"ContainerDied","Data":"a045fed70f1eb7b53b413cc268dd56d9e2c909019a74a3cbdc6c2fb1c1511bd0"}
Sep 14 08:03:33.056697 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.056679    2563 scope.go:117] "RemoveContainer" containerID="efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218"
Sep 14 08:03:33.057045 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:33.057026    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218\": container with ID starting with efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218 not found: ID does not exist" containerID="efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218"
Sep 14 08:03:33.057134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.057058    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218"} err="failed to get container status \"efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218\": rpc error: code = NotFound desc = could not find container \"efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218\": container with ID starting with efb8802e6166a00906d9d74b31cba22f591b0cf6b3a1ad55874d855022ff8218 not found: ID does not exist"
Sep 14 08:03:33.057134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.057080    2563 scope.go:117] "RemoveContainer" containerID="e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995"
Sep 14 08:03:33.069183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.069160    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"]
Sep 14 08:03:33.070445 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.070416    2563 scope.go:117] "RemoveContainer" containerID="e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995"
Sep 14 08:03:33.070771 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:33.070715    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995\": container with ID starting with e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995 not found: ID does not exist" containerID="e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995"
Sep 14 08:03:33.070863 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.070778    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995"} err="failed to get container status \"e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995\": rpc error: code = NotFound desc = could not find container \"e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995\": container with ID starting with e26107cf2bf73cef373f2f93dc16676c84df60770f7fcdd2a7570eaf7d6ed995 not found: ID does not exist"
Sep 14 08:03:33.070863 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.070802    2563 scope.go:117] "RemoveContainer" containerID="358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee"
Sep 14 08:03:33.077081 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.077057    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9ab35f8f-7cbfd4558b-p59dz"]
Sep 14 08:03:33.079712 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.079683    2563 scope.go:117] "RemoveContainer" containerID="358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee"
Sep 14 08:03:33.080001 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:03:33.079969    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee\": container with ID starting with 358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee not found: ID does not exist" containerID="358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee"
Sep 14 08:03:33.080058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.080008    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee"} err="failed to get container status \"358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee\": rpc error: code = NotFound desc = could not find container \"358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee\": container with ID starting with 358d3ae77ed6e2493ee9f7cf0f48f827069b8eeb553b79d15ed83b960b47e5ee not found: ID does not exist"
Sep 14 08:03:33.087246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.087224    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"]
Sep 14 08:03:33.092126 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.092107    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-9ab35f8f-5766dc86d7-m6294"]
Sep 14 08:03:33.104134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.104115    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"]
Sep 14 08:03:33.108037 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.108018    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9ab35f8f-6bf8cb6ff8-skgxg"]
Sep 14 08:03:33.432437 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432407    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"]
Sep 14 08:03:33.432843 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432829    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d350e0df-712f-41e1-ae55-c0855b7317a2" containerName="payload-pre-processing"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432844    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d350e0df-712f-41e1-ae55-c0855b7317a2" containerName="payload-pre-processing"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432853    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67" containerName="revoke-keys"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432859    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67" containerName="revoke-keys"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432878    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" containerName="maas-api"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432883    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" containerName="maas-api"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432898    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5950856a-3e78-44df-afd6-b0ced841440e" containerName="payload-processing"
Sep 14 08:03:33.432902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432904    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5950856a-3e78-44df-afd6-b0ced841440e" containerName="payload-processing"
Sep 14 08:03:33.433148 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432960    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5950856a-3e78-44df-afd6-b0ced841440e" containerName="payload-processing"
Sep 14 08:03:33.433148 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432967    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" containerName="maas-api"
Sep 14 08:03:33.433148 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432974    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecbcdd8c-52e6-41ac-a2ee-c60dfd691c67" containerName="revoke-keys"
Sep 14 08:03:33.433148 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.432980    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d350e0df-712f-41e1-ae55-c0855b7317a2" containerName="payload-pre-processing"
Sep 14 08:03:33.437577 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.437558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.440195 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.440174    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 08:03:33.440301 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.440274    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c3bb3b8f-gw-openshift-default-dockercfg-w555x\""
Sep 14 08:03:33.449694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.449670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"]
Sep 14 08:03:33.539035 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.538996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istiod-ca-cert\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-envoy\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-podinfo\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhjcd\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-kube-api-access-qhjcd\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-credential-socket\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539342 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-token\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539342 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539286    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-socket\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-data\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.539411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.539372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-certs\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640325 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhjcd\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-kube-api-access-qhjcd\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640325 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-credential-socket\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-token\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-socket\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-data\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-certs\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istiod-ca-cert\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-envoy\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640775    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-podinfo\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-credential-socket\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.640987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-data\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.641158 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.640970    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-socket\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.641336 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.641304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istiod-ca-cert\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.641449 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.641335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-certs\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.643247 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.643226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-envoy\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.643336 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.643269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-podinfo\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.647910 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.647883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhjcd\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-kube-api-access-qhjcd\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.648249 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.648228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-token\") pod \"e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") " pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:33.748559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:33.748491    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:34.023220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:34.023197    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"]
Sep 14 08:03:34.024546 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:34.024522    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab5ad6b_43d9_42ca_b469_0dbda10073ed.slice/crio-84eaee73f3cf484b6e52db7e5a30b3d3439e0927fd02d1528dd6ee371d33b7bf WatchSource:0}: Error finding container 84eaee73f3cf484b6e52db7e5a30b3d3439e0927fd02d1528dd6ee371d33b7bf: Status 404 returned error can't find the container with id 84eaee73f3cf484b6e52db7e5a30b3d3439e0927fd02d1528dd6ee371d33b7bf
Sep 14 08:03:34.058021 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:34.057992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" event={"ID":"eab5ad6b-43d9-42ca-b469-0dbda10073ed","Type":"ContainerStarted","Data":"84eaee73f3cf484b6e52db7e5a30b3d3439e0927fd02d1528dd6ee371d33b7bf"}
Sep 14 08:03:34.790173 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:34.790135    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5950856a-3e78-44df-afd6-b0ced841440e" path="/var/lib/kubelet/pods/5950856a-3e78-44df-afd6-b0ced841440e/volumes"
Sep 14 08:03:34.790571 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:34.790554    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d350e0df-712f-41e1-ae55-c0855b7317a2" path="/var/lib/kubelet/pods/d350e0df-712f-41e1-ae55-c0855b7317a2/volumes"
Sep 14 08:03:34.790985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:34.790968    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf023a5-82bc-4c6e-aba0-8bc10a99b34d" path="/var/lib/kubelet/pods/dbf023a5-82bc-4c6e-aba0-8bc10a99b34d/volumes"
Sep 14 08:03:36.866183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:36.866143    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:36.866471 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:36.866217    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:36.866471 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:36.866245    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:37.081274 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:37.081233    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" event={"ID":"eab5ad6b-43d9-42ca-b469-0dbda10073ed","Type":"ContainerStarted","Data":"1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"}
Sep 14 08:03:37.102885 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:37.102841    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podStartSLOduration=1.263733591 podStartE2EDuration="4.10282527s" totalImagesPullingTime="2.839091679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:37.101626524 +0000 UTC m=+872.809640663" watchObservedRunningTime="2026-09-14 08:03:37.10282527 +0000 UTC m=+872.810839412"
Sep 14 08:03:37.749119 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:37.749088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:37.754269 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:37.754244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:38.087490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:38.087414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:38.088450 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:38.088429    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:41.693634 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.693599    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"]
Sep 14 08:03:41.697530 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.697509    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.699816 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.699796    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-c3bb3b8f\""
Sep 14 08:03:41.699948 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.699929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-c3bb3b8f-dockercfg-28rcc\""
Sep 14 08:03:41.707198 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.707176    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"]
Sep 14 08:03:41.724109 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.724079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.724218 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.724130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwqhs\" (UniqueName: \"kubernetes.io/projected/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-kube-api-access-jwqhs\") pod \"payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.744238 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.744211    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"]
Sep 14 08:03:41.747853 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.747837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:41.757926 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.757902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"]
Sep 14 08:03:41.825206 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.825177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca6bf39-6690-4004-927d-f5d7892183ac-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") " pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:41.825370 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.825245    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.825370 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.825321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwqhs\" (UniqueName: \"kubernetes.io/projected/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-kube-api-access-jwqhs\") pod \"payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.825461 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.825374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgmzw\" (UniqueName: \"kubernetes.io/projected/9ca6bf39-6690-4004-927d-f5d7892183ac-kube-api-access-pgmzw\") pod \"payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") " pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:41.826085 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.826066    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.832586 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.832565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwqhs\" (UniqueName: \"kubernetes.io/projected/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-kube-api-access-jwqhs\") pod \"payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:41.926844 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.926810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca6bf39-6690-4004-927d-f5d7892183ac-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") " pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:41.927004 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.926917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgmzw\" (UniqueName: \"kubernetes.io/projected/9ca6bf39-6690-4004-927d-f5d7892183ac-kube-api-access-pgmzw\") pod \"payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") " pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:41.927551 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.927527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca6bf39-6690-4004-927d-f5d7892183ac-plugins-config-volume\") pod \"payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") " pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:41.935170 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:41.935146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgmzw\" (UniqueName: \"kubernetes.io/projected/9ca6bf39-6690-4004-927d-f5d7892183ac-kube-api-access-pgmzw\") pod \"payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") " pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:42.007924 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:42.007851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:42.059126 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:42.058930    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:42.278039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:42.278011    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"]
Sep 14 08:03:42.279366 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:42.279340    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd3384e_6d5b_4bc7_97bb_1dc9aa916eb4.slice/crio-c27d452291cf94ef3bb59018597b1295a10fecd177c1e4734e0adecfabce7299 WatchSource:0}: Error finding container c27d452291cf94ef3bb59018597b1295a10fecd177c1e4734e0adecfabce7299: Status 404 returned error can't find the container with id c27d452291cf94ef3bb59018597b1295a10fecd177c1e4734e0adecfabce7299
Sep 14 08:03:42.335901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:42.335879    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"]
Sep 14 08:03:42.337189 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:03:42.337163    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca6bf39_6690_4004_927d_f5d7892183ac.slice/crio-1dd3dcca8743c3dab91d199de8661383b7a3fce5cd95d4226e93f9a1d6023691 WatchSource:0}: Error finding container 1dd3dcca8743c3dab91d199de8661383b7a3fce5cd95d4226e93f9a1d6023691: Status 404 returned error can't find the container with id 1dd3dcca8743c3dab91d199de8661383b7a3fce5cd95d4226e93f9a1d6023691
Sep 14 08:03:43.122601 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.122562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc" event={"ID":"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4","Type":"ContainerStarted","Data":"02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773"}
Sep 14 08:03:43.123209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.122628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc" event={"ID":"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4","Type":"ContainerStarted","Data":"c27d452291cf94ef3bb59018597b1295a10fecd177c1e4734e0adecfabce7299"}
Sep 14 08:03:43.123209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.122760    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:43.124343 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.124320    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s" event={"ID":"9ca6bf39-6690-4004-927d-f5d7892183ac","Type":"ContainerStarted","Data":"3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d"}
Sep 14 08:03:43.124492 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.124358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s" event={"ID":"9ca6bf39-6690-4004-927d-f5d7892183ac","Type":"ContainerStarted","Data":"1dd3dcca8743c3dab91d199de8661383b7a3fce5cd95d4226e93f9a1d6023691"}
Sep 14 08:03:43.124492 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.124419    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:43.162332 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.162269    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc" podStartSLOduration=2.162252701 podStartE2EDuration="2.162252701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:43.143425134 +0000 UTC m=+878.851439274" watchObservedRunningTime="2026-09-14 08:03:43.162252701 +0000 UTC m=+878.870266844"
Sep 14 08:03:43.162481 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:43.162436    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s" podStartSLOduration=2.162426318 podStartE2EDuration="2.162426318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:43.160411723 +0000 UTC m=+878.868425859" watchObservedRunningTime="2026-09-14 08:03:43.162426318 +0000 UTC m=+878.870440458"
Sep 14 08:03:54.129941 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:54.129898    2563 patch_prober.go:28] interesting pod/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:03:54.130394 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:54.129973    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:03:54.131767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:54.131693    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:03:54.131924 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:54.131910    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:03:55.204256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:55.204226    2563 generic.go:356] "Generic (PLEG): container finished" podID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerID="1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045" exitCode=0
Sep 14 08:03:55.204650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:55.204298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" event={"ID":"eab5ad6b-43d9-42ca-b469-0dbda10073ed","Type":"ContainerDied","Data":"1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"}
Sep 14 08:03:55.204792 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:55.204776    2563 scope.go:117] "RemoveContainer" containerID="1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"
Sep 14 08:03:55.206941 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:55.206913    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:55.207039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:55.206980    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:55.207039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:55.207013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:56.211424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:56.211390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" event={"ID":"eab5ad6b-43d9-42ca-b469-0dbda10073ed","Type":"ContainerStarted","Data":"0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614"}
Sep 14 08:03:56.749434 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:56.749397    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:03:56.749637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:56.749518    2563 patch_prober.go:28] interesting pod/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:03:56.749637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:56.749565    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 14 08:03:57.749715 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:57.749675    2563 patch_prober.go:28] interesting pod/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:03:57.750207 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:57.749767    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 14 08:03:58.749083 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:58.749032    2563 patch_prober.go:28] interesting pod/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:03:58.749293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:58.749121    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 14 08:03:59.749249 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:59.749212    2563 patch_prober.go:28] interesting pod/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:03:59.749849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:03:59.749814    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 14 08:04:00.749330 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:00.749290    2563 patch_prober.go:28] interesting pod/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:04:00.749942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:00.749907    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 14 08:04:01.752315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:01.752286    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:04:01.752688 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:01.752553    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:04:01.753248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:01.753229    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:04:03.690335 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.690297    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"]
Sep 14 08:04:03.694036 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.694014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:03.696437 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.696413    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:04:03.696546 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.696502    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:04:03.700662 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.700639    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"]
Sep 14 08:04:03.842494 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.842453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae891d3-8e58-4796-a13a-2c074cfe585e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc\" (UID: \"aae891d3-8e58-4796-a13a-2c074cfe585e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:03.943901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.943805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae891d3-8e58-4796-a13a-2c074cfe585e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc\" (UID: \"aae891d3-8e58-4796-a13a-2c074cfe585e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:03.944450 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:03.944424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae891d3-8e58-4796-a13a-2c074cfe585e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc\" (UID: \"aae891d3-8e58-4796-a13a-2c074cfe585e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:04.005375 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:04.005337    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:04.273885 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:04.273858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"]
Sep 14 08:04:04.275767 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:04:04.275716    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae891d3_8e58_4796_a13a_2c074cfe585e.slice/crio-8ce903bdcbb18be98283dc0f8b29e630e0b549bb22c61aadd9981d2bcaafa41b WatchSource:0}: Error finding container 8ce903bdcbb18be98283dc0f8b29e630e0b549bb22c61aadd9981d2bcaafa41b: Status 404 returned error can't find the container with id 8ce903bdcbb18be98283dc0f8b29e630e0b549bb22c61aadd9981d2bcaafa41b
Sep 14 08:04:04.731409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:04.731381    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:04:04.731854 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:04.731837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:04:05.263694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:05.263657    2563 generic.go:356] "Generic (PLEG): container finished" podID="aae891d3-8e58-4796-a13a-2c074cfe585e" containerID="a06c40478aeeb64288a5e8510c3ed86d5244243a83373f12cd727053fd13abc1" exitCode=0
Sep 14 08:04:05.263893 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:05.263756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc" event={"ID":"aae891d3-8e58-4796-a13a-2c074cfe585e","Type":"ContainerDied","Data":"a06c40478aeeb64288a5e8510c3ed86d5244243a83373f12cd727053fd13abc1"}
Sep 14 08:04:05.263893 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:05.263813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc" event={"ID":"aae891d3-8e58-4796-a13a-2c074cfe585e","Type":"ContainerStarted","Data":"8ce903bdcbb18be98283dc0f8b29e630e0b549bb22c61aadd9981d2bcaafa41b"}
Sep 14 08:04:06.402313 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:06.402286    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:06.469767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:06.469721    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae891d3-8e58-4796-a13a-2c074cfe585e-maas-api-service-ca\") pod \"aae891d3-8e58-4796-a13a-2c074cfe585e\" (UID: \"aae891d3-8e58-4796-a13a-2c074cfe585e\") "
Sep 14 08:04:06.470053 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:06.470031    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aae891d3-8e58-4796-a13a-2c074cfe585e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aae891d3-8e58-4796-a13a-2c074cfe585e" (UID: "aae891d3-8e58-4796-a13a-2c074cfe585e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:06.570656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:06.570588    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aae891d3-8e58-4796-a13a-2c074cfe585e-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:07.275534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:07.275501    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"
Sep 14 08:04:07.275534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:07.275511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc" event={"ID":"aae891d3-8e58-4796-a13a-2c074cfe585e","Type":"ContainerDied","Data":"8ce903bdcbb18be98283dc0f8b29e630e0b549bb22c61aadd9981d2bcaafa41b"}
Sep 14 08:04:07.275772 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:07.275563    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ce903bdcbb18be98283dc0f8b29e630e0b549bb22c61aadd9981d2bcaafa41b"
Sep 14 08:04:10.466138 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.466104    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"]
Sep 14 08:04:10.466618 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.466304    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s" podUID="9ca6bf39-6690-4004-927d-f5d7892183ac" containerName="payload-processing" containerID="cri-o://3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d" gracePeriod=30
Sep 14 08:04:10.491371 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.488322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"]
Sep 14 08:04:10.491371 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.489074    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc" podUID="1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" containerName="payload-pre-processing" containerID="cri-o://02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773" gracePeriod=30
Sep 14 08:04:10.617187 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.617153    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"]
Sep 14 08:04:10.623455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.623425    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-c3bb3b8f-v9zpc"]
Sep 14 08:04:10.750882 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.750852    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"]
Sep 14 08:04:10.751484 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.751068    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy" containerID="cri-o://0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614" gracePeriod=30
Sep 14 08:04:10.763170 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.763143    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:04:10.787179 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.787154    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:04:10.789992 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.789967    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aae891d3-8e58-4796-a13a-2c074cfe585e" path="/var/lib/kubelet/pods/aae891d3-8e58-4796-a13a-2c074cfe585e/volumes"
Sep 14 08:04:10.912691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.912659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca6bf39-6690-4004-927d-f5d7892183ac-plugins-config-volume\") pod \"9ca6bf39-6690-4004-927d-f5d7892183ac\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") "
Sep 14 08:04:10.912905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.912708    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgmzw\" (UniqueName: \"kubernetes.io/projected/9ca6bf39-6690-4004-927d-f5d7892183ac-kube-api-access-pgmzw\") pod \"9ca6bf39-6690-4004-927d-f5d7892183ac\" (UID: \"9ca6bf39-6690-4004-927d-f5d7892183ac\") "
Sep 14 08:04:10.912905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.912849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-plugins-config-volume\") pod \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") "
Sep 14 08:04:10.912905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.912885    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwqhs\" (UniqueName: \"kubernetes.io/projected/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-kube-api-access-jwqhs\") pod \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\" (UID: \"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4\") "
Sep 14 08:04:10.913176 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.913144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca6bf39-6690-4004-927d-f5d7892183ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ca6bf39-6690-4004-927d-f5d7892183ac" (UID: "9ca6bf39-6690-4004-927d-f5d7892183ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:10.913266 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.913246    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" (UID: "1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:10.914979 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.914955    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca6bf39-6690-4004-927d-f5d7892183ac-kube-api-access-pgmzw" (OuterVolumeSpecName: "kube-api-access-pgmzw") pod "9ca6bf39-6690-4004-927d-f5d7892183ac" (UID: "9ca6bf39-6690-4004-927d-f5d7892183ac"). InnerVolumeSpecName "kube-api-access-pgmzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:10.915063 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:10.914992    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-kube-api-access-jwqhs" (OuterVolumeSpecName: "kube-api-access-jwqhs") pod "1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" (UID: "1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4"). InnerVolumeSpecName "kube-api-access-jwqhs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:11.014002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.013932    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:11.014002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.013956    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwqhs\" (UniqueName: \"kubernetes.io/projected/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4-kube-api-access-jwqhs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:11.014002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.013968    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca6bf39-6690-4004-927d-f5d7892183ac-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:11.014002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.013977    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgmzw\" (UniqueName: \"kubernetes.io/projected/9ca6bf39-6690-4004-927d-f5d7892183ac-kube-api-access-pgmzw\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:11.299569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.299477    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ca6bf39-6690-4004-927d-f5d7892183ac" containerID="3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d" exitCode=0
Sep 14 08:04:11.299569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.299525    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"
Sep 14 08:04:11.299828 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.299566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s" event={"ID":"9ca6bf39-6690-4004-927d-f5d7892183ac","Type":"ContainerDied","Data":"3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d"}
Sep 14 08:04:11.299828 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.299622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s" event={"ID":"9ca6bf39-6690-4004-927d-f5d7892183ac","Type":"ContainerDied","Data":"1dd3dcca8743c3dab91d199de8661383b7a3fce5cd95d4226e93f9a1d6023691"}
Sep 14 08:04:11.299828 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.299648    2563 scope.go:117] "RemoveContainer" containerID="3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d"
Sep 14 08:04:11.301002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.300980    2563 generic.go:356] "Generic (PLEG): container finished" podID="1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" containerID="02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773" exitCode=0
Sep 14 08:04:11.301095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.301015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc" event={"ID":"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4","Type":"ContainerDied","Data":"02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773"}
Sep 14 08:04:11.301095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.301023    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"
Sep 14 08:04:11.301095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.301052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc" event={"ID":"1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4","Type":"ContainerDied","Data":"c27d452291cf94ef3bb59018597b1295a10fecd177c1e4734e0adecfabce7299"}
Sep 14 08:04:11.309457 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.309438    2563 scope.go:117] "RemoveContainer" containerID="3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d"
Sep 14 08:04:11.309762 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:11.309741    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d\": container with ID starting with 3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d not found: ID does not exist" containerID="3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d"
Sep 14 08:04:11.309842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.309769    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d"} err="failed to get container status \"3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d\": rpc error: code = NotFound desc = could not find container \"3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d\": container with ID starting with 3c6878c41790f35395d3b7a6bb68df27d04bae30400eef810b9e7c90844acb6d not found: ID does not exist"
Sep 14 08:04:11.309842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.309789    2563 scope.go:117] "RemoveContainer" containerID="02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773"
Sep 14 08:04:11.317985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.317965    2563 scope.go:117] "RemoveContainer" containerID="02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773"
Sep 14 08:04:11.318225 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:11.318206    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773\": container with ID starting with 02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773 not found: ID does not exist" containerID="02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773"
Sep 14 08:04:11.318267 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.318234    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773"} err="failed to get container status \"02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773\": rpc error: code = NotFound desc = could not find container \"02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773\": container with ID starting with 02e3306a2b50119908e0bc18198a46d5f20a483f78dd5919754c3d6422cd3773 not found: ID does not exist"
Sep 14 08:04:11.324176 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.324154    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"]
Sep 14 08:04:11.327353 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.327331    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-c3bb3b8f-55cbb7d646-5wzsc"]
Sep 14 08:04:11.341512 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.341491    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"]
Sep 14 08:04:11.346128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:11.346108    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-c3bb3b8f-f7bb5b7c4-nm47s"]
Sep 14 08:04:12.789113 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:12.789084    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" path="/var/lib/kubelet/pods/1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4/volumes"
Sep 14 08:04:12.789471 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:12.789417    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca6bf39-6690-4004-927d-f5d7892183ac" path="/var/lib/kubelet/pods/9ca6bf39-6690-4004-927d-f5d7892183ac/volumes"
Sep 14 08:04:16.012657 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.012633    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:04:16.168445 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168368    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-token\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168455    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-podinfo\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhjcd\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-kube-api-access-qhjcd\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168507    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-credential-socket\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-data\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168602    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-socket\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-certs\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168687    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istiod-ca-cert\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.168845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168776    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-envoy\") pod \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\" (UID: \"eab5ad6b-43d9-42ca-b469-0dbda10073ed\") "
Sep 14 08:04:16.169036 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.168979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:16.169095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.169022    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:16.169095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.169082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:16.169205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.169157    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.169205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.169178    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-workload-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.169312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.169233    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:16.169348 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.169326    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:16.170790 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.170761    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:16.170978 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.170927    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:04:16.171068 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.171040    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-kube-api-access-qhjcd" (OuterVolumeSpecName: "kube-api-access-qhjcd") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "kube-api-access-qhjcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:16.171146 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.171132    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eab5ad6b-43d9-42ca-b469-0dbda10073ed" (UID: "eab5ad6b-43d9-42ca-b469-0dbda10073ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:16.270135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270110    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-podinfo\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.270135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270131    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhjcd\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-kube-api-access-qhjcd\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.270290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270142    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-credential-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.270290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270152    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-data\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.270290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270160    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istiod-ca-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.270290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270167    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-envoy\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.270290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.270174    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab5ad6b-43d9-42ca-b469-0dbda10073ed-istio-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.332844 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.332814    2563 generic.go:356] "Generic (PLEG): container finished" podID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerID="0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614" exitCode=0
Sep 14 08:04:16.332966 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.332864    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"
Sep 14 08:04:16.332966 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.332869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" event={"ID":"eab5ad6b-43d9-42ca-b469-0dbda10073ed","Type":"ContainerDied","Data":"0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614"}
Sep 14 08:04:16.332966 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.332912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6" event={"ID":"eab5ad6b-43d9-42ca-b469-0dbda10073ed","Type":"ContainerDied","Data":"84eaee73f3cf484b6e52db7e5a30b3d3439e0927fd02d1528dd6ee371d33b7bf"}
Sep 14 08:04:16.332966 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.332933    2563 scope.go:117] "RemoveContainer" containerID="0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614"
Sep 14 08:04:16.342541 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.342524    2563 scope.go:117] "RemoveContainer" containerID="1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"
Sep 14 08:04:16.350706 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.350682    2563 scope.go:117] "RemoveContainer" containerID="0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614"
Sep 14 08:04:16.351003 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:16.350977    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614\": container with ID starting with 0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614 not found: ID does not exist" containerID="0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614"
Sep 14 08:04:16.351079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.351016    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614"} err="failed to get container status \"0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614\": rpc error: code = NotFound desc = could not find container \"0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614\": container with ID starting with 0d23a2713036197ec1764766bbc2e64836a53f47e4a310646c2d8d3bfe2f4614 not found: ID does not exist"
Sep 14 08:04:16.351079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.351038    2563 scope.go:117] "RemoveContainer" containerID="1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"
Sep 14 08:04:16.351394 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:16.351365    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045\": container with ID starting with 1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045 not found: ID does not exist" containerID="1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"
Sep 14 08:04:16.351492 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.351437    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045"} err="failed to get container status \"1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045\": rpc error: code = NotFound desc = could not find container \"1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045\": container with ID starting with 1b907da9147fa621109374a285017ae02ce246eac5867e4c2d7de8e0287c5045 not found: ID does not exist"
Sep 14 08:04:16.354373 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.354349    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"]
Sep 14 08:04:16.358294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.358267    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c3bb3b8f-gw-openshift-default-5bf6764bcb-t92j6"]
Sep 14 08:04:16.792024 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:16.791993    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" path="/var/lib/kubelet/pods/eab5ad6b-43d9-42ca-b469-0dbda10073ed/volumes"
Sep 14 08:04:18.505281 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"]
Sep 14 08:04:18.505699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505683    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy"
Sep 14 08:04:18.505756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505702    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy"
Sep 14 08:04:18.505756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505716    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca6bf39-6690-4004-927d-f5d7892183ac" containerName="payload-processing"
Sep 14 08:04:18.505756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505736    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca6bf39-6690-4004-927d-f5d7892183ac" containerName="payload-processing"
Sep 14 08:04:18.505756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505743    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae891d3-8e58-4796-a13a-2c074cfe585e" containerName="revoke-keys"
Sep 14 08:04:18.505756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505751    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae891d3-8e58-4796-a13a-2c074cfe585e" containerName="revoke-keys"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505767    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505776    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505790    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" containerName="payload-pre-processing"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505795    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" containerName="payload-pre-processing"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505864    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505872    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca6bf39-6690-4004-927d-f5d7892183ac" containerName="payload-processing"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505877    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eab5ad6b-43d9-42ca-b469-0dbda10073ed" containerName="istio-proxy"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505883    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aae891d3-8e58-4796-a13a-2c074cfe585e" containerName="revoke-keys"
Sep 14 08:04:18.505916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.505891    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fd3384e-6d5b-4bc7-97bb-1dc9aa916eb4" containerName="payload-pre-processing"
Sep 14 08:04:18.510785 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.510763    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.513192 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.513170    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:04:18.514103 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.514048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ddcb78c8-serving-cert\""
Sep 14 08:04:18.514214 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.514061    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:04:18.514214 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.514143    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:04:18.514214 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.514208    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:04:18.519065 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.519044    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"]
Sep 14 08:04:18.589076 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.589039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.589225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.589083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a8a2f-bec8-4898-9b01-0127241a1f81-openshift-service-ca\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.589225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.589107    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-metrics-certs\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.589225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.589129    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tff2\" (UniqueName: \"kubernetes.io/projected/4b6a8a2f-bec8-4898-9b01-0127241a1f81-kube-api-access-7tff2\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.690069 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.690037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.690228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.690088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a8a2f-bec8-4898-9b01-0127241a1f81-openshift-service-ca\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.690228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.690123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-metrics-certs\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.690228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.690158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tff2\" (UniqueName: \"kubernetes.io/projected/4b6a8a2f-bec8-4898-9b01-0127241a1f81-kube-api-access-7tff2\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.690228 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:18.690192    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-serving-cert: secret "maas-api-e2e-ait-ddcb78c8-serving-cert" not found
Sep 14 08:04:18.690446 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:18.690261    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls podName:4b6a8a2f-bec8-4898-9b01-0127241a1f81 nodeName:}" failed. No retries permitted until 2026-09-14 08:04:19.190239801 +0000 UTC m=+914.898253937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls") pod "maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" (UID: "4b6a8a2f-bec8-4898-9b01-0127241a1f81") : secret "maas-api-e2e-ait-ddcb78c8-serving-cert" not found
Sep 14 08:04:18.690845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.690823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a8a2f-bec8-4898-9b01-0127241a1f81-openshift-service-ca\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.692531 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.692514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-metrics-certs\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:18.698379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:18.698354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tff2\" (UniqueName: \"kubernetes.io/projected/4b6a8a2f-bec8-4898-9b01-0127241a1f81-kube-api-access-7tff2\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:19.175921 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.175888    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"]
Sep 14 08:04:19.179880 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.179857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.181992 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.181972    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ddcb78c8\""
Sep 14 08:04:19.190313 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.190290    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"]
Sep 14 08:04:19.194854 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.194833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb9d7a3-d97b-4d32-a335-5b48370b639b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.194941 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.194863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7kk\" (UniqueName: \"kubernetes.io/projected/2bb9d7a3-d97b-4d32-a335-5b48370b639b-kube-api-access-qh7kk\") pod \"payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.194998 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.194964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:19.197278 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.197253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls\") pod \"maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:19.224104 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.224078    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"]
Sep 14 08:04:19.227601 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.227586    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.229685 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.229663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ddcb78c8-dockercfg-5j9tl\""
Sep 14 08:04:19.237391 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.237371    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"]
Sep 14 08:04:19.295619 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.295586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb9d7a3-d97b-4d32-a335-5b48370b639b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.295768 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.295630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7mj\" (UniqueName: \"kubernetes.io/projected/2f37ca6b-3571-4ae3-97dd-c412396d916d-kube-api-access-cb7mj\") pod \"payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") " pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.295768 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.295754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7kk\" (UniqueName: \"kubernetes.io/projected/2bb9d7a3-d97b-4d32-a335-5b48370b639b-kube-api-access-qh7kk\") pod \"payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.295893 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.295861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f37ca6b-3571-4ae3-97dd-c412396d916d-plugins-config-volume\") pod \"payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") " pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.296255 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.296236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb9d7a3-d97b-4d32-a335-5b48370b639b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.303278 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.303249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7kk\" (UniqueName: \"kubernetes.io/projected/2bb9d7a3-d97b-4d32-a335-5b48370b639b-kube-api-access-qh7kk\") pod \"payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.396394 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.396371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f37ca6b-3571-4ae3-97dd-c412396d916d-plugins-config-volume\") pod \"payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") " pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.396551 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.396446    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7mj\" (UniqueName: \"kubernetes.io/projected/2f37ca6b-3571-4ae3-97dd-c412396d916d-kube-api-access-cb7mj\") pod \"payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") " pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.400166 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.397609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f37ca6b-3571-4ae3-97dd-c412396d916d-plugins-config-volume\") pod \"payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") " pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.408048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.408023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7mj\" (UniqueName: \"kubernetes.io/projected/2f37ca6b-3571-4ae3-97dd-c412396d916d-kube-api-access-cb7mj\") pod \"payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") " pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.424702 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.424680    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:19.491484 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.491220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:19.539572 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.539234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:19.698807 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.698777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"]
Sep 14 08:04:19.700490 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:04:19.700459    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6a8a2f_bec8_4898_9b01_0127241a1f81.slice/crio-a78656bdf4e80e12926f32d8bf77b15bc140864a853efaccf90ab6ce84800535 WatchSource:0}: Error finding container a78656bdf4e80e12926f32d8bf77b15bc140864a853efaccf90ab6ce84800535: Status 404 returned error can't find the container with id a78656bdf4e80e12926f32d8bf77b15bc140864a853efaccf90ab6ce84800535
Sep 14 08:04:19.763122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.763094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"]
Sep 14 08:04:19.764801 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:04:19.764776    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb9d7a3_d97b_4d32_a335_5b48370b639b.slice/crio-8512df5f0bee56b87c77ff302e62a03dbee53aca2b2ef48175eb6e74ad7f9c80 WatchSource:0}: Error finding container 8512df5f0bee56b87c77ff302e62a03dbee53aca2b2ef48175eb6e74ad7f9c80: Status 404 returned error can't find the container with id 8512df5f0bee56b87c77ff302e62a03dbee53aca2b2ef48175eb6e74ad7f9c80
Sep 14 08:04:19.844902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:19.844876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"]
Sep 14 08:04:19.846830 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:04:19.846802    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f37ca6b_3571_4ae3_97dd_c412396d916d.slice/crio-e974c08c7d55cadb6e242625d9a5d4e73fc376387e010b117062642968c73b75 WatchSource:0}: Error finding container e974c08c7d55cadb6e242625d9a5d4e73fc376387e010b117062642968c73b75: Status 404 returned error can't find the container with id e974c08c7d55cadb6e242625d9a5d4e73fc376387e010b117062642968c73b75
Sep 14 08:04:20.361109 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.361074    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q" event={"ID":"2f37ca6b-3571-4ae3-97dd-c412396d916d","Type":"ContainerStarted","Data":"dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3"}
Sep 14 08:04:20.361291 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.361134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q" event={"ID":"2f37ca6b-3571-4ae3-97dd-c412396d916d","Type":"ContainerStarted","Data":"e974c08c7d55cadb6e242625d9a5d4e73fc376387e010b117062642968c73b75"}
Sep 14 08:04:20.361291 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.361165    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:20.362892 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.362867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm" event={"ID":"2bb9d7a3-d97b-4d32-a335-5b48370b639b","Type":"ContainerStarted","Data":"bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601"}
Sep 14 08:04:20.363011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.362918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:20.363011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.362941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm" event={"ID":"2bb9d7a3-d97b-4d32-a335-5b48370b639b","Type":"ContainerStarted","Data":"8512df5f0bee56b87c77ff302e62a03dbee53aca2b2ef48175eb6e74ad7f9c80"}
Sep 14 08:04:20.364271 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.364241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" event={"ID":"4b6a8a2f-bec8-4898-9b01-0127241a1f81","Type":"ContainerStarted","Data":"a78656bdf4e80e12926f32d8bf77b15bc140864a853efaccf90ab6ce84800535"}
Sep 14 08:04:20.377392 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.377342    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q" podStartSLOduration=1.377327735 podStartE2EDuration="1.377327735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:20.376271478 +0000 UTC m=+916.084285618" watchObservedRunningTime="2026-09-14 08:04:20.377327735 +0000 UTC m=+916.085341875"
Sep 14 08:04:20.391195 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:20.391153    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm" podStartSLOduration=1.391141515 podStartE2EDuration="1.391141515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:20.390021813 +0000 UTC m=+916.098035964" watchObservedRunningTime="2026-09-14 08:04:20.391141515 +0000 UTC m=+916.099155653"
Sep 14 08:04:21.371156 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:21.371122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" event={"ID":"4b6a8a2f-bec8-4898-9b01-0127241a1f81","Type":"ContainerStarted","Data":"5d4c8793c104b7ccff43ef1b227b0a30a0e5d71b0310b30dd0e177431e8d9b50"}
Sep 14 08:04:21.371596 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:21.371495    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:21.388755 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:21.388695    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" podStartSLOduration=2.47301699 podStartE2EDuration="3.388682563s" totalImagesPullingTime="915.665573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:21.386391103 +0000 UTC m=+917.094405243" watchObservedRunningTime="2026-09-14 08:04:21.388682563 +0000 UTC m=+917.096696702"
Sep 14 08:04:27.383167 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:27.383137    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:31.373423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:31.373390    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:31.373864 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:31.373474    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:45.572011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.571978    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"]
Sep 14 08:04:45.575907 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.575891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:45.578238 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.578218    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:04:45.583125 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.583098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"]
Sep 14 08:04:45.642303 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.642265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q\" (UID: \"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:45.743244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.743208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q\" (UID: \"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:45.743842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.743824    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q\" (UID: \"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:45.888901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:45.888831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:46.162058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:46.162035    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"]
Sep 14 08:04:46.163907 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:04:46.163881    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9695cf_a2e5_4c52_bda9_8ef47d69b11e.slice/crio-48e1e2ed935a21942b42fda8d07b9ba82dee6441e79bcbf2e4211188dade9bd3 WatchSource:0}: Error finding container 48e1e2ed935a21942b42fda8d07b9ba82dee6441e79bcbf2e4211188dade9bd3: Status 404 returned error can't find the container with id 48e1e2ed935a21942b42fda8d07b9ba82dee6441e79bcbf2e4211188dade9bd3
Sep 14 08:04:46.518930 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:46.518897    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q" event={"ID":"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e","Type":"ContainerStarted","Data":"919657e5bce63799e0aa8db1176ac7450070e3fe4339cca694c0f4e53daec8c2"}
Sep 14 08:04:46.519079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:46.518950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q" event={"ID":"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e","Type":"ContainerStarted","Data":"48e1e2ed935a21942b42fda8d07b9ba82dee6441e79bcbf2e4211188dade9bd3"}
Sep 14 08:04:46.534044 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:46.533996    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q" podStartSLOduration=1.533982008 podStartE2EDuration="1.533982008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:46.533314116 +0000 UTC m=+942.241328258" watchObservedRunningTime="2026-09-14 08:04:46.533982008 +0000 UTC m=+942.241996146"
Sep 14 08:04:47.526469 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:47.526432    2563 generic.go:356] "Generic (PLEG): container finished" podID="8c9695cf-a2e5-4c52-bda9-8ef47d69b11e" containerID="919657e5bce63799e0aa8db1176ac7450070e3fe4339cca694c0f4e53daec8c2" exitCode=0
Sep 14 08:04:47.526895 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:47.526513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q" event={"ID":"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e","Type":"ContainerDied","Data":"919657e5bce63799e0aa8db1176ac7450070e3fe4339cca694c0f4e53daec8c2"}
Sep 14 08:04:48.667634 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:48.667609    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:48.774056 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:48.774025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e-maas-api-service-ca\") pod \"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e\" (UID: \"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e\") "
Sep 14 08:04:48.774385 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:48.774360    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8c9695cf-a2e5-4c52-bda9-8ef47d69b11e" (UID: "8c9695cf-a2e5-4c52-bda9-8ef47d69b11e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:48.876099 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:48.876060    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:49.538211 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:49.538167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q" event={"ID":"8c9695cf-a2e5-4c52-bda9-8ef47d69b11e","Type":"ContainerDied","Data":"48e1e2ed935a21942b42fda8d07b9ba82dee6441e79bcbf2e4211188dade9bd3"}
Sep 14 08:04:49.538386 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:49.538247    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48e1e2ed935a21942b42fda8d07b9ba82dee6441e79bcbf2e4211188dade9bd3"
Sep 14 08:04:49.538386 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:49.538191    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"
Sep 14 08:04:56.409137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.409105    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"]
Sep 14 08:04:56.409583 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.409305    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" podUID="4b6a8a2f-bec8-4898-9b01-0127241a1f81" containerName="maas-api" containerID="cri-o://5d4c8793c104b7ccff43ef1b227b0a30a0e5d71b0310b30dd0e177431e8d9b50" gracePeriod=30
Sep 14 08:04:56.464412 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.464380    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"]
Sep 14 08:04:56.464643 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.464612    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q" podUID="2f37ca6b-3571-4ae3-97dd-c412396d916d" containerName="payload-processing" containerID="cri-o://dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3" gracePeriod=30
Sep 14 08:04:56.470657 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.470598    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"]
Sep 14 08:04:56.471008 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.470980    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm" podUID="2bb9d7a3-d97b-4d32-a335-5b48370b639b" containerName="payload-pre-processing" containerID="cri-o://bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601" gracePeriod=30
Sep 14 08:04:56.580329 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.580296    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b6a8a2f-bec8-4898-9b01-0127241a1f81" containerID="5d4c8793c104b7ccff43ef1b227b0a30a0e5d71b0310b30dd0e177431e8d9b50" exitCode=0
Sep 14 08:04:56.580435 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.580359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" event={"ID":"4b6a8a2f-bec8-4898-9b01-0127241a1f81","Type":"ContainerDied","Data":"5d4c8793c104b7ccff43ef1b227b0a30a0e5d71b0310b30dd0e177431e8d9b50"}
Sep 14 08:04:56.713525 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.713495    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"]
Sep 14 08:04:56.717332 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.716621    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ddcb78c8-hsq7q"]
Sep 14 08:04:56.746244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.746218    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:56.749273 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.749251    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:56.760499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.760481    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:56.791877 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.791846    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9695cf-a2e5-4c52-bda9-8ef47d69b11e" path="/var/lib/kubelet/pods/8c9695cf-a2e5-4c52-bda9-8ef47d69b11e/volumes"
Sep 14 08:04:56.854864 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.854834    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls\") pod \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") "
Sep 14 08:04:56.855034 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.854868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-metrics-certs\") pod \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") "
Sep 14 08:04:56.855034 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.854921    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f37ca6b-3571-4ae3-97dd-c412396d916d-plugins-config-volume\") pod \"2f37ca6b-3571-4ae3-97dd-c412396d916d\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") "
Sep 14 08:04:56.855034 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.854954    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb7mj\" (UniqueName: \"kubernetes.io/projected/2f37ca6b-3571-4ae3-97dd-c412396d916d-kube-api-access-cb7mj\") pod \"2f37ca6b-3571-4ae3-97dd-c412396d916d\" (UID: \"2f37ca6b-3571-4ae3-97dd-c412396d916d\") "
Sep 14 08:04:56.855034 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb9d7a3-d97b-4d32-a335-5b48370b639b-plugins-config-volume\") pod \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") "
Sep 14 08:04:56.855248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh7kk\" (UniqueName: \"kubernetes.io/projected/2bb9d7a3-d97b-4d32-a335-5b48370b639b-kube-api-access-qh7kk\") pod \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\" (UID: \"2bb9d7a3-d97b-4d32-a335-5b48370b639b\") "
Sep 14 08:04:56.855248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a8a2f-bec8-4898-9b01-0127241a1f81-openshift-service-ca\") pod \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") "
Sep 14 08:04:56.855248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tff2\" (UniqueName: \"kubernetes.io/projected/4b6a8a2f-bec8-4898-9b01-0127241a1f81-kube-api-access-7tff2\") pod \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\" (UID: \"4b6a8a2f-bec8-4898-9b01-0127241a1f81\") "
Sep 14 08:04:56.855504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb9d7a3-d97b-4d32-a335-5b48370b639b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bb9d7a3-d97b-4d32-a335-5b48370b639b" (UID: "2bb9d7a3-d97b-4d32-a335-5b48370b639b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:56.855504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855468    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f37ca6b-3571-4ae3-97dd-c412396d916d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f37ca6b-3571-4ae3-97dd-c412396d916d" (UID: "2f37ca6b-3571-4ae3-97dd-c412396d916d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:56.855504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b6a8a2f-bec8-4898-9b01-0127241a1f81-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b6a8a2f-bec8-4898-9b01-0127241a1f81" (UID: "4b6a8a2f-bec8-4898-9b01-0127241a1f81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:56.855767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855618    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f37ca6b-3571-4ae3-97dd-c412396d916d-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.855767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855643    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb9d7a3-d97b-4d32-a335-5b48370b639b-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.855767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.855657    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b6a8a2f-bec8-4898-9b01-0127241a1f81-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.857208 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.857184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f37ca6b-3571-4ae3-97dd-c412396d916d-kube-api-access-cb7mj" (OuterVolumeSpecName: "kube-api-access-cb7mj") pod "2f37ca6b-3571-4ae3-97dd-c412396d916d" (UID: "2f37ca6b-3571-4ae3-97dd-c412396d916d"). InnerVolumeSpecName "kube-api-access-cb7mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:56.857399 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.857379    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4b6a8a2f-bec8-4898-9b01-0127241a1f81" (UID: "4b6a8a2f-bec8-4898-9b01-0127241a1f81"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:56.857467 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.857419    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b6a8a2f-bec8-4898-9b01-0127241a1f81" (UID: "4b6a8a2f-bec8-4898-9b01-0127241a1f81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:56.857855 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.857836    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6a8a2f-bec8-4898-9b01-0127241a1f81-kube-api-access-7tff2" (OuterVolumeSpecName: "kube-api-access-7tff2") pod "4b6a8a2f-bec8-4898-9b01-0127241a1f81" (UID: "4b6a8a2f-bec8-4898-9b01-0127241a1f81"). InnerVolumeSpecName "kube-api-access-7tff2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:56.857904 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.857862    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bb9d7a3-d97b-4d32-a335-5b48370b639b-kube-api-access-qh7kk" (OuterVolumeSpecName: "kube-api-access-qh7kk") pod "2bb9d7a3-d97b-4d32-a335-5b48370b639b" (UID: "2bb9d7a3-d97b-4d32-a335-5b48370b639b"). InnerVolumeSpecName "kube-api-access-qh7kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:56.956982 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.956951    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.956982 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.956981    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b6a8a2f-bec8-4898-9b01-0127241a1f81-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.957181 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.956997    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb7mj\" (UniqueName: \"kubernetes.io/projected/2f37ca6b-3571-4ae3-97dd-c412396d916d-kube-api-access-cb7mj\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.957181 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.957013    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh7kk\" (UniqueName: \"kubernetes.io/projected/2bb9d7a3-d97b-4d32-a335-5b48370b639b-kube-api-access-qh7kk\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:56.957181 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:56.957027    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tff2\" (UniqueName: \"kubernetes.io/projected/4b6a8a2f-bec8-4898-9b01-0127241a1f81-kube-api-access-7tff2\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:04:57.587929 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.587893    2563 generic.go:356] "Generic (PLEG): container finished" podID="2bb9d7a3-d97b-4d32-a335-5b48370b639b" containerID="bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601" exitCode=0
Sep 14 08:04:57.587929 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.587931    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"
Sep 14 08:04:57.588353 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.587971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm" event={"ID":"2bb9d7a3-d97b-4d32-a335-5b48370b639b","Type":"ContainerDied","Data":"bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601"}
Sep 14 08:04:57.588353 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.588021    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm" event={"ID":"2bb9d7a3-d97b-4d32-a335-5b48370b639b","Type":"ContainerDied","Data":"8512df5f0bee56b87c77ff302e62a03dbee53aca2b2ef48175eb6e74ad7f9c80"}
Sep 14 08:04:57.588353 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.588056    2563 scope.go:117] "RemoveContainer" containerID="bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601"
Sep 14 08:04:57.589439 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.589362    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"
Sep 14 08:04:57.589439 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.589395    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48" event={"ID":"4b6a8a2f-bec8-4898-9b01-0127241a1f81","Type":"ContainerDied","Data":"a78656bdf4e80e12926f32d8bf77b15bc140864a853efaccf90ab6ce84800535"}
Sep 14 08:04:57.590963 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.590934    2563 generic.go:356] "Generic (PLEG): container finished" podID="2f37ca6b-3571-4ae3-97dd-c412396d916d" containerID="dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3" exitCode=0
Sep 14 08:04:57.591105 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.590977    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"
Sep 14 08:04:57.591105 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.591020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q" event={"ID":"2f37ca6b-3571-4ae3-97dd-c412396d916d","Type":"ContainerDied","Data":"dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3"}
Sep 14 08:04:57.591105 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.591065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q" event={"ID":"2f37ca6b-3571-4ae3-97dd-c412396d916d","Type":"ContainerDied","Data":"e974c08c7d55cadb6e242625d9a5d4e73fc376387e010b117062642968c73b75"}
Sep 14 08:04:57.601696 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.601676    2563 scope.go:117] "RemoveContainer" containerID="bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601"
Sep 14 08:04:57.602164 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:57.602144    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601\": container with ID starting with bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601 not found: ID does not exist" containerID="bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601"
Sep 14 08:04:57.602246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.602176    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601"} err="failed to get container status \"bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601\": rpc error: code = NotFound desc = could not find container \"bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601\": container with ID starting with bfa6dd60ea1728a131d5c8f54295fe53a5ce5ead9310712a4b53217c70ba1601 not found: ID does not exist"
Sep 14 08:04:57.602246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.602199    2563 scope.go:117] "RemoveContainer" containerID="5d4c8793c104b7ccff43ef1b227b0a30a0e5d71b0310b30dd0e177431e8d9b50"
Sep 14 08:04:57.610670 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.610654    2563 scope.go:117] "RemoveContainer" containerID="dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3"
Sep 14 08:04:57.617864 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.617840    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"]
Sep 14 08:04:57.618848 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.618833    2563 scope.go:117] "RemoveContainer" containerID="dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3"
Sep 14 08:04:57.619084 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:04:57.619067    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3\": container with ID starting with dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3 not found: ID does not exist" containerID="dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3"
Sep 14 08:04:57.619131 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.619091    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3"} err="failed to get container status \"dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3\": rpc error: code = NotFound desc = could not find container \"dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3\": container with ID starting with dbf338bd08b8782a499a4ba30f9009b11cf7569e0f4ba60dde69bedf5e3babd3 not found: ID does not exist"
Sep 14 08:04:57.623379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.623358    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ddcb78c8-5558f799d5-h9w48"]
Sep 14 08:04:57.632869 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.632849    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"]
Sep 14 08:04:57.636674 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.636653    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ddcb78c8-6bb4c9f75d-h2zjm"]
Sep 14 08:04:57.650555 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.650513    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"]
Sep 14 08:04:57.654193 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:57.654171    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ddcb78c8-557cf67669-hh77q"]
Sep 14 08:04:58.713615 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.713570    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"]
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714030    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9695cf-a2e5-4c52-bda9-8ef47d69b11e" containerName="revoke-keys"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714042    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9695cf-a2e5-4c52-bda9-8ef47d69b11e" containerName="revoke-keys"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714062    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb9d7a3-d97b-4d32-a335-5b48370b639b" containerName="payload-pre-processing"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714068    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb9d7a3-d97b-4d32-a335-5b48370b639b" containerName="payload-pre-processing"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714074    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f37ca6b-3571-4ae3-97dd-c412396d916d" containerName="payload-processing"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714080    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f37ca6b-3571-4ae3-97dd-c412396d916d" containerName="payload-processing"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714087    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6a8a2f-bec8-4898-9b01-0127241a1f81" containerName="maas-api"
Sep 14 08:04:58.714089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714093    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6a8a2f-bec8-4898-9b01-0127241a1f81" containerName="maas-api"
Sep 14 08:04:58.714499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714153    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6a8a2f-bec8-4898-9b01-0127241a1f81" containerName="maas-api"
Sep 14 08:04:58.714499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714162    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb9d7a3-d97b-4d32-a335-5b48370b639b" containerName="payload-pre-processing"
Sep 14 08:04:58.714499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714169    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9695cf-a2e5-4c52-bda9-8ef47d69b11e" containerName="revoke-keys"
Sep 14 08:04:58.714499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.714174    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f37ca6b-3571-4ae3-97dd-c412396d916d" containerName="payload-processing"
Sep 14 08:04:58.718690 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.718669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.721039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.721017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 08:04:58.721152 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.721017    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8e5c0736-openshift-default-dockercfg-c99d2\""
Sep 14 08:04:58.730617 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.730594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"]
Sep 14 08:04:58.774360 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-data\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-token\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-certs\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-envoy\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774606 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istiod-ca-cert\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774606 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-credential-socket\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774606 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-podinfo\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-socket\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.774714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.774658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vklrz\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-kube-api-access-vklrz\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.789829 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.789803    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bb9d7a3-d97b-4d32-a335-5b48370b639b" path="/var/lib/kubelet/pods/2bb9d7a3-d97b-4d32-a335-5b48370b639b/volumes"
Sep 14 08:04:58.790166 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.790152    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f37ca6b-3571-4ae3-97dd-c412396d916d" path="/var/lib/kubelet/pods/2f37ca6b-3571-4ae3-97dd-c412396d916d/volumes"
Sep 14 08:04:58.790510 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.790498    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6a8a2f-bec8-4898-9b01-0127241a1f81" path="/var/lib/kubelet/pods/4b6a8a2f-bec8-4898-9b01-0127241a1f81/volumes"
Sep 14 08:04:58.875246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-token\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875344 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-certs\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875344 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-envoy\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875344 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istiod-ca-cert\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875485 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-credential-socket\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875485 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-podinfo\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875485 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-socket\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vklrz\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-kube-api-access-vklrz\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-data\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-certs\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.875974 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875947    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-data\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.876070 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.875987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-credential-socket\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.876070 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.876016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istiod-ca-cert\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.876070 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.876048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-socket\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.877678 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.877658    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-envoy\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.877837 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.877818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-podinfo\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.882454 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.882430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-token\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:58.882645 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:58.882628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vklrz\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-kube-api-access-vklrz\") pod \"e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") " pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:59.029922 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.029857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:04:59.305959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.305930    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"]
Sep 14 08:04:59.307445 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:04:59.307411    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d12bd97_33eb_4110_bd3d_7d8c559ac693.slice/crio-c5eeac7ef37d2a5eaaa11b1a6a6aefea9bea810cef1379ae378953a57718c457 WatchSource:0}: Error finding container c5eeac7ef37d2a5eaaa11b1a6a6aefea9bea810cef1379ae378953a57718c457: Status 404 returned error can't find the container with id c5eeac7ef37d2a5eaaa11b1a6a6aefea9bea810cef1379ae378953a57718c457
Sep 14 08:04:59.309743 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.309696    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:59.309843 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.309802    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:59.309903 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.309848    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:59.607529 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.607446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" event={"ID":"9d12bd97-33eb-4110-bd3d-7d8c559ac693","Type":"ContainerStarted","Data":"ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"}
Sep 14 08:04:59.607529 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.607496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" event={"ID":"9d12bd97-33eb-4110-bd3d-7d8c559ac693","Type":"ContainerStarted","Data":"c5eeac7ef37d2a5eaaa11b1a6a6aefea9bea810cef1379ae378953a57718c457"}
Sep 14 08:04:59.626293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:04:59.626239    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podStartSLOduration=1.6262242040000001 podStartE2EDuration="1.626224204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:59.624720118 +0000 UTC m=+955.332734258" watchObservedRunningTime="2026-09-14 08:04:59.626224204 +0000 UTC m=+955.334238342"
Sep 14 08:05:00.029993 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:00.029956    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:00.035374 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:00.035347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:00.613226 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:00.613194    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:00.614345 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:00.614318    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:05.501566 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.501534    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"]
Sep 14 08:05:05.506895 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.506873    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.509284 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.509261    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:05:05.509414 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.509264    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-8e5c0736-serving-cert\""
Sep 14 08:05:05.510198 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.510167    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:05:05.510310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.510217    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:05:05.510310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.510285    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:05:05.516353 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.516331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"]
Sep 14 08:05:05.542018 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.541989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkc5z\" (UniqueName: \"kubernetes.io/projected/8d657fd7-8a74-43c2-8322-8cfcee08e41a-kube-api-access-wkc5z\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.542155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.542057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d657fd7-8a74-43c2-8322-8cfcee08e41a-openshift-service-ca\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.542155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.542130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.542274 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.542166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-metrics-certs\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.643317 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.643289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkc5z\" (UniqueName: \"kubernetes.io/projected/8d657fd7-8a74-43c2-8322-8cfcee08e41a-kube-api-access-wkc5z\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.643317 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.643332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d657fd7-8a74-43c2-8322-8cfcee08e41a-openshift-service-ca\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.643569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.643374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.643569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.643398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-metrics-certs\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.643569 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:05.643497    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-serving-cert: secret "maas-api-e2e-derive-8e5c0736-serving-cert" not found
Sep 14 08:05:05.643743 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:05.643577    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls podName:8d657fd7-8a74-43c2-8322-8cfcee08e41a nodeName:}" failed. No retries permitted until 2026-09-14 08:05:06.143557765 +0000 UTC m=+961.851571897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls") pod "maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" (UID: "8d657fd7-8a74-43c2-8322-8cfcee08e41a") : secret "maas-api-e2e-derive-8e5c0736-serving-cert" not found
Sep 14 08:05:05.644162 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.644143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d657fd7-8a74-43c2-8322-8cfcee08e41a-openshift-service-ca\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.646129 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.646111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-metrics-certs\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:05.654571 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:05.654549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkc5z\" (UniqueName: \"kubernetes.io/projected/8d657fd7-8a74-43c2-8322-8cfcee08e41a-kube-api-access-wkc5z\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:06.093771 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.093736    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"]
Sep 14 08:05:06.097360 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.097345    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.099781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.099759    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-8e5c0736-dockercfg-6k24l\""
Sep 14 08:05:06.099781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.099771    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-8e5c0736\""
Sep 14 08:05:06.109812 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.109788    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"]
Sep 14 08:05:06.129954 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.129927    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"]
Sep 14 08:05:06.133956 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.133941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.140956 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.140931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"]
Sep 14 08:05:06.148899 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.148872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfjnr\" (UniqueName: \"kubernetes.io/projected/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-kube-api-access-bfjnr\") pod \"payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.149000 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.148964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.149039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.149014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:06.152243 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.152217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls\") pod \"maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:06.249971 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.249941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfjnr\" (UniqueName: \"kubernetes.io/projected/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-kube-api-access-bfjnr\") pod \"payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.250081 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.249996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.250081 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.250061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44face2-797c-4819-9831-5c22a416298b-plugins-config-volume\") pod \"payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") " pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.250166 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.250097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq5l2\" (UniqueName: \"kubernetes.io/projected/d44face2-797c-4819-9831-5c22a416298b-kube-api-access-qq5l2\") pod \"payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") " pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.250686 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.250668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.257762 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.257719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfjnr\" (UniqueName: \"kubernetes.io/projected/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-kube-api-access-bfjnr\") pod \"payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.351070 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.351000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44face2-797c-4819-9831-5c22a416298b-plugins-config-volume\") pod \"payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") " pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.351070 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.351050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq5l2\" (UniqueName: \"kubernetes.io/projected/d44face2-797c-4819-9831-5c22a416298b-kube-api-access-qq5l2\") pod \"payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") " pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.351800 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.351782    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44face2-797c-4819-9831-5c22a416298b-plugins-config-volume\") pod \"payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") " pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.358961 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.358938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq5l2\" (UniqueName: \"kubernetes.io/projected/d44face2-797c-4819-9831-5c22a416298b-kube-api-access-qq5l2\") pod \"payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") " pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.408370 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.408349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:06.419466 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.419408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:06.447855 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.447495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:06.685250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.685228    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"]
Sep 14 08:05:06.687134 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:05:06.687110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4fb47e_c9b5_4a80_86dc_ed18825bb2c0.slice/crio-60fa438c165323167d5bef22691eca147cbf70d81b02c37caadf01abf3d4c1ad WatchSource:0}: Error finding container 60fa438c165323167d5bef22691eca147cbf70d81b02c37caadf01abf3d4c1ad: Status 404 returned error can't find the container with id 60fa438c165323167d5bef22691eca147cbf70d81b02c37caadf01abf3d4c1ad
Sep 14 08:05:06.703466 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.703440    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"]
Sep 14 08:05:06.703814 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:05:06.703789    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d657fd7_8a74_43c2_8322_8cfcee08e41a.slice/crio-4926ae0b1b90e7a0f8bf495d237e9656439b7e10054dc4c3a1553bb528644a20 WatchSource:0}: Error finding container 4926ae0b1b90e7a0f8bf495d237e9656439b7e10054dc4c3a1553bb528644a20: Status 404 returned error can't find the container with id 4926ae0b1b90e7a0f8bf495d237e9656439b7e10054dc4c3a1553bb528644a20
Sep 14 08:05:06.705138 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.705121    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:05:06.740352 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:05:06.740311    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44face2_797c_4819_9831_5c22a416298b.slice/crio-c2be503fe1eccd6c975ff82a68be066efdca72c913604e082d62a065bc56c448 WatchSource:0}: Error finding container c2be503fe1eccd6c975ff82a68be066efdca72c913604e082d62a065bc56c448: Status 404 returned error can't find the container with id c2be503fe1eccd6c975ff82a68be066efdca72c913604e082d62a065bc56c448
Sep 14 08:05:06.745356 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:06.745319    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"]
Sep 14 08:05:07.655612 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.655573    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p" event={"ID":"d44face2-797c-4819-9831-5c22a416298b","Type":"ContainerStarted","Data":"800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7"}
Sep 14 08:05:07.655821 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.655638    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p" event={"ID":"d44face2-797c-4819-9831-5c22a416298b","Type":"ContainerStarted","Data":"c2be503fe1eccd6c975ff82a68be066efdca72c913604e082d62a065bc56c448"}
Sep 14 08:05:07.655821 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.655692    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:07.657494 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.657201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7" event={"ID":"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0","Type":"ContainerStarted","Data":"5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e"}
Sep 14 08:05:07.657494 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.657252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:07.657494 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.657270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7" event={"ID":"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0","Type":"ContainerStarted","Data":"60fa438c165323167d5bef22691eca147cbf70d81b02c37caadf01abf3d4c1ad"}
Sep 14 08:05:07.659038 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.659009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" event={"ID":"8d657fd7-8a74-43c2-8322-8cfcee08e41a","Type":"ContainerStarted","Data":"2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1"}
Sep 14 08:05:07.659129 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.659052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" event={"ID":"8d657fd7-8a74-43c2-8322-8cfcee08e41a","Type":"ContainerStarted","Data":"4926ae0b1b90e7a0f8bf495d237e9656439b7e10054dc4c3a1553bb528644a20"}
Sep 14 08:05:07.659173 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.659147    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:07.673446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.673407    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p" podStartSLOduration=1.6733956079999999 podStartE2EDuration="1.673395608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:07.670429302 +0000 UTC m=+963.378443440" watchObservedRunningTime="2026-09-14 08:05:07.673395608 +0000 UTC m=+963.381409748"
Sep 14 08:05:07.687317 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.687277    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" podStartSLOduration=2.3030760949999998 podStartE2EDuration="2.6872635s" totalImagesPullingTime="384.187405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:07.686216511 +0000 UTC m=+963.394230648" watchObservedRunningTime="2026-09-14 08:05:07.6872635 +0000 UTC m=+963.395277702"
Sep 14 08:05:07.702456 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:07.702418    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7" podStartSLOduration=1.702406242 podStartE2EDuration="1.702406242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:07.700543999 +0000 UTC m=+963.408558152" watchObservedRunningTime="2026-09-14 08:05:07.702406242 +0000 UTC m=+963.410420380"
Sep 14 08:05:13.670124 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:13.670088    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:18.666981 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:18.666948    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:18.667455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:18.667285    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:25.767790 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:25.767755    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerID="ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111" exitCode=0
Sep 14 08:05:25.768209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:25.767824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" event={"ID":"9d12bd97-33eb-4110-bd3d-7d8c559ac693","Type":"ContainerDied","Data":"ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"}
Sep 14 08:05:25.768325 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:25.768309    2563 scope.go:117] "RemoveContainer" containerID="ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"
Sep 14 08:05:25.769483 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:25.769457    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:05:25.769551 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:25.769528    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:05:25.769591 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:25.769562    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:05:26.775200 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:26.775166    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" event={"ID":"9d12bd97-33eb-4110-bd3d-7d8c559ac693","Type":"ContainerStarted","Data":"c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8"}
Sep 14 08:05:27.030069 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:27.029953    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:27.031417 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:27.031389    2563 patch_prober.go:28] interesting pod/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:05:27.031550 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:27.031454    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 08:05:28.031106 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:28.031067    2563 patch_prober.go:28] interesting pod/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:05:28.031569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:28.031532    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 08:05:29.030963 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:29.030919    2563 patch_prober.go:28] interesting pod/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:05:29.031135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:29.030984    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 08:05:30.030969 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.030927    2563 patch_prober.go:28] interesting pod/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:05:30.031185 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.030992    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 08:05:30.613594 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.613544    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:30.979233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.979196    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"]
Sep 14 08:05:30.982030 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.982014    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:30.984221 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.984205    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:05:30.988812 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:30.988787    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"]
Sep 14 08:05:31.010004 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.009978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b8b826-2fa8-4fdd-85e5-65f91de92f1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6\" (UID: \"21b8b826-2fa8-4fdd-85e5-65f91de92f1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:31.030805 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.030780    2563 patch_prober.go:28] interesting pod/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:05:31.030896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.030837    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 14 08:05:31.111337 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.111302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b8b826-2fa8-4fdd-85e5-65f91de92f1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6\" (UID: \"21b8b826-2fa8-4fdd-85e5-65f91de92f1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:31.112010 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.111988    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b8b826-2fa8-4fdd-85e5-65f91de92f1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6\" (UID: \"21b8b826-2fa8-4fdd-85e5-65f91de92f1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:31.295847 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.295760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:31.592953 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.592922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"]
Sep 14 08:05:31.594654 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:05:31.594624    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b8b826_2fa8_4fdd_85e5_65f91de92f1a.slice/crio-91717db8cddb389e2e31a03df11b2638f70b8f463e20e68720e508566b728896 WatchSource:0}: Error finding container 91717db8cddb389e2e31a03df11b2638f70b8f463e20e68720e508566b728896: Status 404 returned error can't find the container with id 91717db8cddb389e2e31a03df11b2638f70b8f463e20e68720e508566b728896
Sep 14 08:05:31.808255 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:31.808214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6" event={"ID":"21b8b826-2fa8-4fdd-85e5-65f91de92f1a","Type":"ContainerStarted","Data":"91717db8cddb389e2e31a03df11b2638f70b8f463e20e68720e508566b728896"}
Sep 14 08:05:32.034663 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:32.034635    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:32.035673 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:32.035651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:32.815611 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:32.815577    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b8b826-2fa8-4fdd-85e5-65f91de92f1a" containerID="b388cf436e843f91362308d9e5eca32eb4c2d0e8133f71c9e3ddaed31b5478c4" exitCode=0
Sep 14 08:05:32.816012 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:32.815684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6" event={"ID":"21b8b826-2fa8-4fdd-85e5-65f91de92f1a","Type":"ContainerDied","Data":"b388cf436e843f91362308d9e5eca32eb4c2d0e8133f71c9e3ddaed31b5478c4"}
Sep 14 08:05:33.957775 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:33.957753    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:34.042833 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:34.042807    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b8b826-2fa8-4fdd-85e5-65f91de92f1a-maas-api-service-ca\") pod \"21b8b826-2fa8-4fdd-85e5-65f91de92f1a\" (UID: \"21b8b826-2fa8-4fdd-85e5-65f91de92f1a\") "
Sep 14 08:05:34.043194 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:34.043163    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b8b826-2fa8-4fdd-85e5-65f91de92f1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21b8b826-2fa8-4fdd-85e5-65f91de92f1a" (UID: "21b8b826-2fa8-4fdd-85e5-65f91de92f1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:34.143825 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:34.143757    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b8b826-2fa8-4fdd-85e5-65f91de92f1a-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:34.829006 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:34.828975    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"
Sep 14 08:05:34.829006 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:34.828993    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6" event={"ID":"21b8b826-2fa8-4fdd-85e5-65f91de92f1a","Type":"ContainerDied","Data":"91717db8cddb389e2e31a03df11b2638f70b8f463e20e68720e508566b728896"}
Sep 14 08:05:34.829227 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:34.829036    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91717db8cddb389e2e31a03df11b2638f70b8f463e20e68720e508566b728896"
Sep 14 08:05:43.027457 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.027426    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"]
Sep 14 08:05:43.027942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.027712    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" podUID="8d657fd7-8a74-43c2-8322-8cfcee08e41a" containerName="maas-api" containerID="cri-o://2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1" gracePeriod=30
Sep 14 08:05:43.069590 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.069561    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"]
Sep 14 08:05:43.069996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.069967    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p" podUID="d44face2-797c-4819-9831-5c22a416298b" containerName="payload-processing" containerID="cri-o://800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7" gracePeriod=30
Sep 14 08:05:43.078244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.078218    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"]
Sep 14 08:05:43.078625 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.078571    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7" podUID="7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" containerName="payload-pre-processing" containerID="cri-o://5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e" gracePeriod=30
Sep 14 08:05:43.310658 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.310605    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"]
Sep 14 08:05:43.316435 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.316408    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-8e5c0736-rmxd6"]
Sep 14 08:05:43.358527 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.358504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:43.361889 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.361861    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:43.379295 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.379271    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:43.427766 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.427711    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-metrics-certs\") pod \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") "
Sep 14 08:05:43.427941 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.427897    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44face2-797c-4819-9831-5c22a416298b-plugins-config-volume\") pod \"d44face2-797c-4819-9831-5c22a416298b\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") "
Sep 14 08:05:43.428010 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.427987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-plugins-config-volume\") pod \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") "
Sep 14 08:05:43.428069 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.428055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkc5z\" (UniqueName: \"kubernetes.io/projected/8d657fd7-8a74-43c2-8322-8cfcee08e41a-kube-api-access-wkc5z\") pod \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") "
Sep 14 08:05:43.428131 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.428085    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls\") pod \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") "
Sep 14 08:05:43.428131 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.428113    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq5l2\" (UniqueName: \"kubernetes.io/projected/d44face2-797c-4819-9831-5c22a416298b-kube-api-access-qq5l2\") pod \"d44face2-797c-4819-9831-5c22a416298b\" (UID: \"d44face2-797c-4819-9831-5c22a416298b\") "
Sep 14 08:05:43.428244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.428175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfjnr\" (UniqueName: \"kubernetes.io/projected/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-kube-api-access-bfjnr\") pod \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\" (UID: \"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0\") "
Sep 14 08:05:43.428300 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.428250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d657fd7-8a74-43c2-8322-8cfcee08e41a-openshift-service-ca\") pod \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\" (UID: \"8d657fd7-8a74-43c2-8322-8cfcee08e41a\") "
Sep 14 08:05:43.428968 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.428937    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d657fd7-8a74-43c2-8322-8cfcee08e41a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d657fd7-8a74-43c2-8322-8cfcee08e41a" (UID: "8d657fd7-8a74-43c2-8322-8cfcee08e41a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:43.429094 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.429063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d44face2-797c-4819-9831-5c22a416298b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d44face2-797c-4819-9831-5c22a416298b" (UID: "d44face2-797c-4819-9831-5c22a416298b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:43.429232 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.429199    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" (UID: "7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:43.432437 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.432336    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d657fd7-8a74-43c2-8322-8cfcee08e41a-kube-api-access-wkc5z" (OuterVolumeSpecName: "kube-api-access-wkc5z") pod "8d657fd7-8a74-43c2-8322-8cfcee08e41a" (UID: "8d657fd7-8a74-43c2-8322-8cfcee08e41a"). InnerVolumeSpecName "kube-api-access-wkc5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:43.432544 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.432431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d44face2-797c-4819-9831-5c22a416298b-kube-api-access-qq5l2" (OuterVolumeSpecName: "kube-api-access-qq5l2") pod "d44face2-797c-4819-9831-5c22a416298b" (UID: "d44face2-797c-4819-9831-5c22a416298b"). InnerVolumeSpecName "kube-api-access-qq5l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:43.432686 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.432652    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d657fd7-8a74-43c2-8322-8cfcee08e41a" (UID: "8d657fd7-8a74-43c2-8322-8cfcee08e41a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:05:43.432888 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.432828    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8d657fd7-8a74-43c2-8322-8cfcee08e41a" (UID: "8d657fd7-8a74-43c2-8322-8cfcee08e41a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:05:43.433031 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.432966    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-kube-api-access-bfjnr" (OuterVolumeSpecName: "kube-api-access-bfjnr") pod "7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" (UID: "7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0"). InnerVolumeSpecName "kube-api-access-bfjnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:43.445808 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.445777    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"]
Sep 14 08:05:43.446071 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.446020    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" containerID="cri-o://c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8" gracePeriod=30
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529335    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d657fd7-8a74-43c2-8322-8cfcee08e41a-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529378    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529391    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d44face2-797c-4819-9831-5c22a416298b-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529405    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529417    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkc5z\" (UniqueName: \"kubernetes.io/projected/8d657fd7-8a74-43c2-8322-8cfcee08e41a-kube-api-access-wkc5z\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529428    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d657fd7-8a74-43c2-8322-8cfcee08e41a-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529442    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq5l2\" (UniqueName: \"kubernetes.io/projected/d44face2-797c-4819-9831-5c22a416298b-kube-api-access-qq5l2\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.529746 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.529453    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfjnr\" (UniqueName: \"kubernetes.io/projected/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0-kube-api-access-bfjnr\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:43.722788 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.722744    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"]
Sep 14 08:05:43.723213 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723197    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d657fd7-8a74-43c2-8322-8cfcee08e41a" containerName="maas-api"
Sep 14 08:05:43.723256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723215    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d657fd7-8a74-43c2-8322-8cfcee08e41a" containerName="maas-api"
Sep 14 08:05:43.723256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723230    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b8b826-2fa8-4fdd-85e5-65f91de92f1a" containerName="revoke-keys"
Sep 14 08:05:43.723256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723235    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b8b826-2fa8-4fdd-85e5-65f91de92f1a" containerName="revoke-keys"
Sep 14 08:05:43.723256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723243    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" containerName="payload-pre-processing"
Sep 14 08:05:43.723256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723249    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" containerName="payload-pre-processing"
Sep 14 08:05:43.723406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723269    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d44face2-797c-4819-9831-5c22a416298b" containerName="payload-processing"
Sep 14 08:05:43.723406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723277    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d44face2-797c-4819-9831-5c22a416298b" containerName="payload-processing"
Sep 14 08:05:43.723406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723342    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d657fd7-8a74-43c2-8322-8cfcee08e41a" containerName="maas-api"
Sep 14 08:05:43.723406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723354    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b8b826-2fa8-4fdd-85e5-65f91de92f1a" containerName="revoke-keys"
Sep 14 08:05:43.723406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723366    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d44face2-797c-4819-9831-5c22a416298b" containerName="payload-processing"
Sep 14 08:05:43.723406 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.723375    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" containerName="payload-pre-processing"
Sep 14 08:05:43.725632 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.725616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:43.728457 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.728439    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:05:43.736528 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.736507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"]
Sep 14 08:05:43.831806 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.831774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e63847f-b785-4516-b80f-d472be61541b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-885430-89xj2\" (UID: \"3e63847f-b785-4516-b80f-d472be61541b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:43.876919 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.876884    2563 generic.go:356] "Generic (PLEG): container finished" podID="d44face2-797c-4819-9831-5c22a416298b" containerID="800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7" exitCode=0
Sep 14 08:05:43.877045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.876930    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"
Sep 14 08:05:43.877045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.876965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p" event={"ID":"d44face2-797c-4819-9831-5c22a416298b","Type":"ContainerDied","Data":"800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7"}
Sep 14 08:05:43.877045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.877014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p" event={"ID":"d44face2-797c-4819-9831-5c22a416298b","Type":"ContainerDied","Data":"c2be503fe1eccd6c975ff82a68be066efdca72c913604e082d62a065bc56c448"}
Sep 14 08:05:43.877045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.877038    2563 scope.go:117] "RemoveContainer" containerID="800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7"
Sep 14 08:05:43.878349 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.878329    2563 generic.go:356] "Generic (PLEG): container finished" podID="7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" containerID="5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e" exitCode=0
Sep 14 08:05:43.878444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.878373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"
Sep 14 08:05:43.878444 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.878412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7" event={"ID":"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0","Type":"ContainerDied","Data":"5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e"}
Sep 14 08:05:43.878566 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.878465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7" event={"ID":"7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0","Type":"ContainerDied","Data":"60fa438c165323167d5bef22691eca147cbf70d81b02c37caadf01abf3d4c1ad"}
Sep 14 08:05:43.880742 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.880027    2563 generic.go:356] "Generic (PLEG): container finished" podID="8d657fd7-8a74-43c2-8322-8cfcee08e41a" containerID="2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1" exitCode=0
Sep 14 08:05:43.880742 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.880097    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"
Sep 14 08:05:43.880742 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.880124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" event={"ID":"8d657fd7-8a74-43c2-8322-8cfcee08e41a","Type":"ContainerDied","Data":"2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1"}
Sep 14 08:05:43.880742 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.880163    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68" event={"ID":"8d657fd7-8a74-43c2-8322-8cfcee08e41a","Type":"ContainerDied","Data":"4926ae0b1b90e7a0f8bf495d237e9656439b7e10054dc4c3a1553bb528644a20"}
Sep 14 08:05:43.887008 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.886991    2563 scope.go:117] "RemoveContainer" containerID="800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7"
Sep 14 08:05:43.887330 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:43.887304    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7\": container with ID starting with 800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7 not found: ID does not exist" containerID="800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7"
Sep 14 08:05:43.887457 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.887428    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7"} err="failed to get container status \"800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7\": rpc error: code = NotFound desc = could not find container \"800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7\": container with ID starting with 800df00ec82a97a95a0dc900d11374ce3c529874b1d5e42ef0e41c803534bef7 not found: ID does not exist"
Sep 14 08:05:43.887457 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.887457    2563 scope.go:117] "RemoveContainer" containerID="5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e"
Sep 14 08:05:43.897180 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.897163    2563 scope.go:117] "RemoveContainer" containerID="5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e"
Sep 14 08:05:43.897448 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:43.897430    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e\": container with ID starting with 5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e not found: ID does not exist" containerID="5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e"
Sep 14 08:05:43.897490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.897454    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e"} err="failed to get container status \"5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e\": rpc error: code = NotFound desc = could not find container \"5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e\": container with ID starting with 5ce4953d19853be804e0539a8a01a99ee0feb1beededbc01acf53a63c5936a1e not found: ID does not exist"
Sep 14 08:05:43.897490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.897468    2563 scope.go:117] "RemoveContainer" containerID="2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1"
Sep 14 08:05:43.901146 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.901124    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"]
Sep 14 08:05:43.905643 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.905620    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8e5c0736-65f65f74f-jwh7p"]
Sep 14 08:05:43.907097 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.907080    2563 scope.go:117] "RemoveContainer" containerID="2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1"
Sep 14 08:05:43.907366 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:43.907346    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1\": container with ID starting with 2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1 not found: ID does not exist" containerID="2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1"
Sep 14 08:05:43.907423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.907373    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1"} err="failed to get container status \"2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1\": rpc error: code = NotFound desc = could not find container \"2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1\": container with ID starting with 2241ae792196195683a5c3408c7cacc6b47a5c612a0c0c8c4854482ca016e5d1 not found: ID does not exist"
Sep 14 08:05:43.914160 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.914138    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"]
Sep 14 08:05:43.917393 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.917370    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-8e5c0736-85757bc49b-2tb68"]
Sep 14 08:05:43.928480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.928462    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"]
Sep 14 08:05:43.931889 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.931872    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8e5c0736-69b85498b6-wb2m7"]
Sep 14 08:05:43.932220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.932201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e63847f-b785-4516-b80f-d472be61541b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-885430-89xj2\" (UID: \"3e63847f-b785-4516-b80f-d472be61541b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:43.932758 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:43.932742    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e63847f-b785-4516-b80f-d472be61541b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-885430-89xj2\" (UID: \"3e63847f-b785-4516-b80f-d472be61541b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:44.036476 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.036406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:44.305580 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.305553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"]
Sep 14 08:05:44.307162 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:05:44.307134    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e63847f_b785_4516_b80f_d472be61541b.slice/crio-dff2a1047ec982a1b118ed85bc9dced49ed53e8550719e67ff327c319641e073 WatchSource:0}: Error finding container dff2a1047ec982a1b118ed85bc9dced49ed53e8550719e67ff327c319641e073: Status 404 returned error can't find the container with id dff2a1047ec982a1b118ed85bc9dced49ed53e8550719e67ff327c319641e073
Sep 14 08:05:44.789860 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.789823    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21b8b826-2fa8-4fdd-85e5-65f91de92f1a" path="/var/lib/kubelet/pods/21b8b826-2fa8-4fdd-85e5-65f91de92f1a/volumes"
Sep 14 08:05:44.790300 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.790269    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0" path="/var/lib/kubelet/pods/7f4fb47e-c9b5-4a80-86dc-ed18825bb2c0/volumes"
Sep 14 08:05:44.790712 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.790693    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d657fd7-8a74-43c2-8322-8cfcee08e41a" path="/var/lib/kubelet/pods/8d657fd7-8a74-43c2-8322-8cfcee08e41a/volumes"
Sep 14 08:05:44.791109 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.791092    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d44face2-797c-4819-9831-5c22a416298b" path="/var/lib/kubelet/pods/d44face2-797c-4819-9831-5c22a416298b/volumes"
Sep 14 08:05:44.889332 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.889301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2" event={"ID":"3e63847f-b785-4516-b80f-d472be61541b","Type":"ContainerStarted","Data":"39f6412dde9b5c7850f745b7c441e0e6aac3a6b54accc2673452d2e3abdbde31"}
Sep 14 08:05:44.889485 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.889349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2" event={"ID":"3e63847f-b785-4516-b80f-d472be61541b","Type":"ContainerStarted","Data":"dff2a1047ec982a1b118ed85bc9dced49ed53e8550719e67ff327c319641e073"}
Sep 14 08:05:44.907122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:44.907079    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2" podStartSLOduration=1.907063696 podStartE2EDuration="1.907063696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:44.905971147 +0000 UTC m=+1000.613985289" watchObservedRunningTime="2026-09-14 08:05:44.907063696 +0000 UTC m=+1000.615077835"
Sep 14 08:05:45.898776 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:45.898741    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e63847f-b785-4516-b80f-d472be61541b" containerID="39f6412dde9b5c7850f745b7c441e0e6aac3a6b54accc2673452d2e3abdbde31" exitCode=0
Sep 14 08:05:45.899225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:45.898823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2" event={"ID":"3e63847f-b785-4516-b80f-d472be61541b","Type":"ContainerDied","Data":"39f6412dde9b5c7850f745b7c441e0e6aac3a6b54accc2673452d2e3abdbde31"}
Sep 14 08:05:47.034948 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.034925    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:47.036964 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.036938    2563 patch_prober.go:28] interesting pod/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:05:47.037060 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.036992    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:05:47.166255 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.166175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e63847f-b785-4516-b80f-d472be61541b-maas-api-service-ca\") pod \"3e63847f-b785-4516-b80f-d472be61541b\" (UID: \"3e63847f-b785-4516-b80f-d472be61541b\") "
Sep 14 08:05:47.166639 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.166611    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e63847f-b785-4516-b80f-d472be61541b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e63847f-b785-4516-b80f-d472be61541b" (UID: "3e63847f-b785-4516-b80f-d472be61541b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:47.266943 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.266898    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e63847f-b785-4516-b80f-d472be61541b-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:47.911716 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.911687    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"
Sep 14 08:05:47.911716 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.911697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2" event={"ID":"3e63847f-b785-4516-b80f-d472be61541b","Type":"ContainerDied","Data":"dff2a1047ec982a1b118ed85bc9dced49ed53e8550719e67ff327c319641e073"}
Sep 14 08:05:47.911961 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:47.911759    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dff2a1047ec982a1b118ed85bc9dced49ed53e8550719e67ff327c319641e073"
Sep 14 08:05:48.694009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.693985    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:48.780262 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780184    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-credential-socket\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780262 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-podinfo\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780262 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780254    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istiod-ca-cert\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780513 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780318    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vklrz\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-kube-api-access-vklrz\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780513 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780345    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-socket\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780513 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780436    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:48.780667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780516    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-envoy\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780566    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-token\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780611    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-data\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:48.780976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780702    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-certs\") pod \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\" (UID: \"9d12bd97-33eb-4110-bd3d-7d8c559ac693\") "
Sep 14 08:05:48.780976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780741    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:48.780976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.780859    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:48.781134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.781052    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:48.781134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.781125    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-data\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.781242 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.781145    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-credential-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.781242 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.781159    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istiod-ca-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.781242 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.781172    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.782742 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.782692    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:05:48.782917 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.782889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:48.782992 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.782897    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:05:48.783047 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.783003    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-kube-api-access-vklrz" (OuterVolumeSpecName: "kube-api-access-vklrz") pod "9d12bd97-33eb-4110-bd3d-7d8c559ac693" (UID: "9d12bd97-33eb-4110-bd3d-7d8c559ac693"). InnerVolumeSpecName "kube-api-access-vklrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:48.882058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.881951    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vklrz\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-kube-api-access-vklrz\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.882058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.881983    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-envoy\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.882058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.881999    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.882058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.882013    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d12bd97-33eb-4110-bd3d-7d8c559ac693-workload-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.882058 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.882028    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d12bd97-33eb-4110-bd3d-7d8c559ac693-istio-podinfo\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:48.919053 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.919020    2563 generic.go:356] "Generic (PLEG): container finished" podID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerID="c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8" exitCode=0
Sep 14 08:05:48.919149 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.919070    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"
Sep 14 08:05:48.919149 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.919112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" event={"ID":"9d12bd97-33eb-4110-bd3d-7d8c559ac693","Type":"ContainerDied","Data":"c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8"}
Sep 14 08:05:48.919223 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.919171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm" event={"ID":"9d12bd97-33eb-4110-bd3d-7d8c559ac693","Type":"ContainerDied","Data":"c5eeac7ef37d2a5eaaa11b1a6a6aefea9bea810cef1379ae378953a57718c457"}
Sep 14 08:05:48.919223 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.919195    2563 scope.go:117] "RemoveContainer" containerID="c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8"
Sep 14 08:05:48.928446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.928431    2563 scope.go:117] "RemoveContainer" containerID="ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"
Sep 14 08:05:48.936111 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.936087    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"]
Sep 14 08:05:48.937448 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.937433    2563 scope.go:117] "RemoveContainer" containerID="c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8"
Sep 14 08:05:48.937718 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:48.937699    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8\": container with ID starting with c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8 not found: ID does not exist" containerID="c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8"
Sep 14 08:05:48.937813 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.937751    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8"} err="failed to get container status \"c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8\": rpc error: code = NotFound desc = could not find container \"c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8\": container with ID starting with c855ed7b86449914f956649a29965f7a5302cbda81fa022d4aa084224d37e1a8 not found: ID does not exist"
Sep 14 08:05:48.937813 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.937779    2563 scope.go:117] "RemoveContainer" containerID="ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"
Sep 14 08:05:48.938092 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:48.938069    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111\": container with ID starting with ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111 not found: ID does not exist" containerID="ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"
Sep 14 08:05:48.938157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.938100    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111"} err="failed to get container status \"ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111\": rpc error: code = NotFound desc = could not find container \"ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111\": container with ID starting with ac46dab46462ba1810c58f4225fc36949cf914036e5ea940cdf9f799df172111 not found: ID does not exist"
Sep 14 08:05:48.939253 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:48.939234    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8e5c0736-openshift-default-6886c7d996-q8kwm"]
Sep 14 08:05:50.788611 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:50.788579    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" path="/var/lib/kubelet/pods/9d12bd97-33eb-4110-bd3d-7d8c559ac693/volumes"
Sep 14 08:05:54.415191 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.415158    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"]
Sep 14 08:05:54.415655 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.415398    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm" podUID="9eb70445-936b-4420-be1e-9c15e7767846" containerName="payload-pre-processing" containerID="cri-o://21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7" gracePeriod=30
Sep 14 08:05:54.554761 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.554715    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"]
Sep 14 08:05:54.556623 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.556602    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-885430-89xj2"]
Sep 14 08:05:54.662595 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.662574    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:05:54.738021 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.737994    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb70445-936b-4420-be1e-9c15e7767846-plugins-config-volume\") pod \"9eb70445-936b-4420-be1e-9c15e7767846\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") "
Sep 14 08:05:54.738202 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.738144    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx4bn\" (UniqueName: \"kubernetes.io/projected/9eb70445-936b-4420-be1e-9c15e7767846-kube-api-access-sx4bn\") pod \"9eb70445-936b-4420-be1e-9c15e7767846\" (UID: \"9eb70445-936b-4420-be1e-9c15e7767846\") "
Sep 14 08:05:54.738428 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.738405    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eb70445-936b-4420-be1e-9c15e7767846-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9eb70445-936b-4420-be1e-9c15e7767846" (UID: "9eb70445-936b-4420-be1e-9c15e7767846"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:54.740223 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.740192    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eb70445-936b-4420-be1e-9c15e7767846-kube-api-access-sx4bn" (OuterVolumeSpecName: "kube-api-access-sx4bn") pod "9eb70445-936b-4420-be1e-9c15e7767846" (UID: "9eb70445-936b-4420-be1e-9c15e7767846"). InnerVolumeSpecName "kube-api-access-sx4bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:54.790090 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.790054    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e63847f-b785-4516-b80f-d472be61541b" path="/var/lib/kubelet/pods/3e63847f-b785-4516-b80f-d472be61541b/volumes"
Sep 14 08:05:54.839875 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.839848    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx4bn\" (UniqueName: \"kubernetes.io/projected/9eb70445-936b-4420-be1e-9c15e7767846-kube-api-access-sx4bn\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.839875 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.839872    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9eb70445-936b-4420-be1e-9c15e7767846-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:05:54.955649 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.955613    2563 generic.go:356] "Generic (PLEG): container finished" podID="9eb70445-936b-4420-be1e-9c15e7767846" containerID="21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7" exitCode=0
Sep 14 08:05:54.955836 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.955675    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"
Sep 14 08:05:54.955836 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.955700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm" event={"ID":"9eb70445-936b-4420-be1e-9c15e7767846","Type":"ContainerDied","Data":"21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7"}
Sep 14 08:05:54.955836 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.955773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm" event={"ID":"9eb70445-936b-4420-be1e-9c15e7767846","Type":"ContainerDied","Data":"5de6ddd1cbc1ae9a84c45f1b229cdb631b8fb67940adffb98f4a635cd2b1ab0f"}
Sep 14 08:05:54.955836 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.955796    2563 scope.go:117] "RemoveContainer" containerID="21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7"
Sep 14 08:05:54.966569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.966533    2563 scope.go:117] "RemoveContainer" containerID="21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7"
Sep 14 08:05:54.966789 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:05:54.966767    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7\": container with ID starting with 21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7 not found: ID does not exist" containerID="21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7"
Sep 14 08:05:54.966857 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.966799    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7"} err="failed to get container status \"21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7\": rpc error: code = NotFound desc = could not find container \"21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7\": container with ID starting with 21de63e79b4d8f957dedf7b95ea1446a7cc5ae00d8c4f7de1d1a257e7449acd7 not found: ID does not exist"
Sep 14 08:05:54.974945 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.974925    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"]
Sep 14 08:05:54.977038 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:54.976989    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-885430-74774bddc4-qfnqm"]
Sep 14 08:05:56.791026 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:05:56.790983    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eb70445-936b-4420-be1e-9c15e7767846" path="/var/lib/kubelet/pods/9eb70445-936b-4420-be1e-9c15e7767846/volumes"
Sep 14 08:06:19.379561 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379523    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"]
Sep 14 08:06:19.379973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379934    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy"
Sep 14 08:06:19.379973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379946    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy"
Sep 14 08:06:19.379973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379957    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy"
Sep 14 08:06:19.379973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379963    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379977    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e63847f-b785-4516-b80f-d472be61541b" containerName="revoke-keys"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.379983    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e63847f-b785-4516-b80f-d472be61541b" containerName="revoke-keys"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.380000    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eb70445-936b-4420-be1e-9c15e7767846" containerName="payload-pre-processing"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.380006    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eb70445-936b-4420-be1e-9c15e7767846" containerName="payload-pre-processing"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.380065    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.380072    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d12bd97-33eb-4110-bd3d-7d8c559ac693" containerName="istio-proxy"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.380077    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eb70445-936b-4420-be1e-9c15e7767846" containerName="payload-pre-processing"
Sep 14 08:06:19.380108 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.380084    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e63847f-b785-4516-b80f-d472be61541b" containerName="revoke-keys"
Sep 14 08:06:19.384638 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.384622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.388007 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.387986    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"default-dockercfg-4p5xb\""
Sep 14 08:06:19.388158 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.388146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"openshift-service-ca.crt\""
Sep 14 08:06:19.388229 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.388177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"premium-w4-cd7582-kserve-self-signed-certs\""
Sep 14 08:06:19.388534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.388514    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"kube-root-ca.crt\""
Sep 14 08:06:19.390712 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.390691    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"]
Sep 14 08:06:19.468409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.468378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-home\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.468409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.468408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-tmp-dir\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.468597 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.468430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkgqt\" (UniqueName: \"kubernetes.io/projected/8540d7c2-55c1-4e92-b260-b00a2ae89591-kube-api-access-fkgqt\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.468597 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.468479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-dshm\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.468597 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.468561    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8540d7c2-55c1-4e92-b260-b00a2ae89591-tls-certs\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.468711 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.468618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-model-cache\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570072 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8540d7c2-55c1-4e92-b260-b00a2ae89591-tls-certs\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-model-cache\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-home\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-tmp-dir\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkgqt\" (UniqueName: \"kubernetes.io/projected/8540d7c2-55c1-4e92-b260-b00a2ae89591-kube-api-access-fkgqt\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-dshm\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570594 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-home\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-tmp-dir\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.570667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.570637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-model-cache\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.572442 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.572425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-dshm\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.572529 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.572513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8540d7c2-55c1-4e92-b260-b00a2ae89591-tls-certs\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.577942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.577919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkgqt\" (UniqueName: \"kubernetes.io/projected/8540d7c2-55c1-4e92-b260-b00a2ae89591-kube-api-access-fkgqt\") pod \"premium-w4-cd7582-kserve-86f769885f-mvlpt\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") " pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.695399 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.695370    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:19.969872 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:19.969850    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"]
Sep 14 08:06:19.971561 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:06:19.971529    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8540d7c2_55c1_4e92_b260_b00a2ae89591.slice/crio-c98c0c4be84c1351e240bd520de7b375f742ed965335ba3dcd83e1568a98ef22 WatchSource:0}: Error finding container c98c0c4be84c1351e240bd520de7b375f742ed965335ba3dcd83e1568a98ef22: Status 404 returned error can't find the container with id c98c0c4be84c1351e240bd520de7b375f742ed965335ba3dcd83e1568a98ef22
Sep 14 08:06:20.098329 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:20.098264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt" event={"ID":"8540d7c2-55c1-4e92-b260-b00a2ae89591","Type":"ContainerStarted","Data":"6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b"}
Sep 14 08:06:20.098329 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:20.098333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt" event={"ID":"8540d7c2-55c1-4e92-b260-b00a2ae89591","Type":"ContainerStarted","Data":"c98c0c4be84c1351e240bd520de7b375f742ed965335ba3dcd83e1568a98ef22"}
Sep 14 08:06:20.117534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:20.117485    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt" podStartSLOduration=1.117469949 podStartE2EDuration="1.117469949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:20.11517232 +0000 UTC m=+1035.823186459" watchObservedRunningTime="2026-09-14 08:06:20.117469949 +0000 UTC m=+1035.825484116"
Sep 14 08:06:29.696076 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:29.696042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:29.696076 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:29.696085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:29.708341 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:29.708317    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:30.161482 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:30.161404    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:06:37.932138 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:37.932103    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"]
Sep 14 08:06:37.935947 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:37.935925    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:37.938433 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:37.938409    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 08:06:37.938559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:37.938456    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-239599-openshift-default-dockercfg-8rv72\""
Sep 14 08:06:37.949842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:37.949814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"]
Sep 14 08:06:38.035085 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-envoy\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-certs\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-data\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-credential-socket\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbzsg\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-kube-api-access-qbzsg\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035346 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-podinfo\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035346 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-token\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035346 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-socket\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.035458 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.035348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istiod-ca-cert\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136608 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-certs\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136759 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-data\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136759 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-credential-socket\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136759 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbzsg\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-kube-api-access-qbzsg\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136759 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-podinfo\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-token\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-socket\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.136984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.136883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istiod-ca-cert\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.137156 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.137047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-envoy\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.137156 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.137116    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-certs\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.137269 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.137195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-credential-socket\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.137269 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.137236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-socket\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.137388 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.137322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-data\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.137536 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.137517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istiod-ca-cert\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.139263 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.139244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-podinfo\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.139536 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.139506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-envoy\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.146388 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.146359    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-token\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.146388 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.146382    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbzsg\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-kube-api-access-qbzsg\") pod \"e2e-explicit-239599-openshift-default-f466b4968-qzbcr\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") " pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.251010 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.250935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:38.525924 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.525899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"]
Sep 14 08:06:38.532428 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.532391    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:38.532526 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.532458    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:38.532526 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:38.532484    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:39.202523 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:39.202490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" event={"ID":"d5deb6c6-62c7-4936-b94d-25b5e92e36f7","Type":"ContainerStarted","Data":"b1af8d49f3a293f76554ecda9e6bea0fb18efbf25807f5811c579c69d098747f"}
Sep 14 08:06:39.202912 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:39.202541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" event={"ID":"d5deb6c6-62c7-4936-b94d-25b5e92e36f7","Type":"ContainerStarted","Data":"ee3d74a69a279b31eec651c3e73589fb4cb2efaaeda529975bc6f9f9287d3a31"}
Sep 14 08:06:39.223323 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:39.223278    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podStartSLOduration=2.223264377 podStartE2EDuration="2.223264377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:39.220698083 +0000 UTC m=+1054.928712221" watchObservedRunningTime="2026-09-14 08:06:39.223264377 +0000 UTC m=+1054.931278515"
Sep 14 08:06:39.251245 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:39.251215    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:39.256361 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:39.256341    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:40.208282 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:40.208252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:40.209296 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:40.209275    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:06:44.046277 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.046235    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"]
Sep 14 08:06:44.050805 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.050781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.053228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.053206    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-239599-serving-cert\""
Sep 14 08:06:44.053342 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.053251    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:06:44.055062 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.054600    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:06:44.055062 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.054647    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:06:44.055062 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.055000    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:06:44.059224 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.059008    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"]
Sep 14 08:06:44.202984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.202955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-metrics-certs\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.203141 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.203003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpdq4\" (UniqueName: \"kubernetes.io/projected/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-kube-api-access-wpdq4\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.203141 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.203042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.203141 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.203079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-openshift-service-ca\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.303766 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.303668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpdq4\" (UniqueName: \"kubernetes.io/projected/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-kube-api-access-wpdq4\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.303766 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.303750    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.303991 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.303786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-openshift-service-ca\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.303991 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.303898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-metrics-certs\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.303991 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:06:44.303904    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-239599-serving-cert: secret "maas-api-e2e-explicit-239599-serving-cert" not found
Sep 14 08:06:44.303991 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:06:44.303972    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls podName:05101f4b-a8dc-48c9-823c-5e6a9ddab7a0 nodeName:}" failed. No retries permitted until 2026-09-14 08:06:44.803951364 +0000 UTC m=+1060.511965501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls") pod "maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" (UID: "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0") : secret "maas-api-e2e-explicit-239599-serving-cert" not found
Sep 14 08:06:44.304438 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.304421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-openshift-service-ca\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.306455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.306434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-metrics-certs\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.314856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.314831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpdq4\" (UniqueName: \"kubernetes.io/projected/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-kube-api-access-wpdq4\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.633517 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.633440    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"]
Sep 14 08:06:44.637165 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.637150    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.639969 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.639945    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-239599\""
Sep 14 08:06:44.640095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.640020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-239599-dockercfg-hmdgx\""
Sep 14 08:06:44.650221 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.650197    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"]
Sep 14 08:06:44.670851 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.670823    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"]
Sep 14 08:06:44.674300 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.674284    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.683780 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.683752    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"]
Sep 14 08:06:44.816207 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.816175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fwjs\" (UniqueName: \"kubernetes.io/projected/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-kube-api-access-4fwjs\") pod \"payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.816397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.816256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.816397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.816303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.816397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.816335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a10822-c5f6-42c8-bda9-b60fac1a8237-plugins-config-volume\") pod \"payload-processing-e2e-explicit-239599-bf8fc869c-m96vd\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") " pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.816397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.816395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdsg2\" (UniqueName: \"kubernetes.io/projected/04a10822-c5f6-42c8-bda9-b60fac1a8237-kube-api-access-kdsg2\") pod \"payload-processing-e2e-explicit-239599-bf8fc869c-m96vd\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") " pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.818686 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.818663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls\") pod \"maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.917534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.917471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.917534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.917520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a10822-c5f6-42c8-bda9-b60fac1a8237-plugins-config-volume\") pod \"payload-processing-e2e-explicit-239599-bf8fc869c-m96vd\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") " pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.917767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.917570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdsg2\" (UniqueName: \"kubernetes.io/projected/04a10822-c5f6-42c8-bda9-b60fac1a8237-kube-api-access-kdsg2\") pod \"payload-processing-e2e-explicit-239599-bf8fc869c-m96vd\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") " pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.917767 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.917604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fwjs\" (UniqueName: \"kubernetes.io/projected/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-kube-api-access-4fwjs\") pod \"payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.918182 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.918162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.918336 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.918318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a10822-c5f6-42c8-bda9-b60fac1a8237-plugins-config-volume\") pod \"payload-processing-e2e-explicit-239599-bf8fc869c-m96vd\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") " pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.925772 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.925752    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdsg2\" (UniqueName: \"kubernetes.io/projected/04a10822-c5f6-42c8-bda9-b60fac1a8237-kube-api-access-kdsg2\") pod \"payload-processing-e2e-explicit-239599-bf8fc869c-m96vd\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") " pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:44.925864 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.925807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fwjs\" (UniqueName: \"kubernetes.io/projected/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-kube-api-access-4fwjs\") pod \"payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.947943 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.947920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:44.964414 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.964395    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:44.985460 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:44.985436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:45.230210 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:45.230157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"]
Sep 14 08:06:45.231306 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:06:45.231277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19eea12_9ca8_40ea_b3ba_a916d0e9891a.slice/crio-5c5a1b44e5c3a65f4ab6b03e9e6397ec3e00a9b353cec8aa17e1204fc78e1242 WatchSource:0}: Error finding container 5c5a1b44e5c3a65f4ab6b03e9e6397ec3e00a9b353cec8aa17e1204fc78e1242: Status 404 returned error can't find the container with id 5c5a1b44e5c3a65f4ab6b03e9e6397ec3e00a9b353cec8aa17e1204fc78e1242
Sep 14 08:06:45.257256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:45.257236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"]
Sep 14 08:06:45.262795 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:06:45.262775    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05101f4b_a8dc_48c9_823c_5e6a9ddab7a0.slice/crio-fbd8273b5e5b87675e5e05b5f87c6a5fc9861e27eadab2166e1cd7b9c30ebbf4 WatchSource:0}: Error finding container fbd8273b5e5b87675e5e05b5f87c6a5fc9861e27eadab2166e1cd7b9c30ebbf4: Status 404 returned error can't find the container with id fbd8273b5e5b87675e5e05b5f87c6a5fc9861e27eadab2166e1cd7b9c30ebbf4
Sep 14 08:06:45.285318 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:45.285296    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"]
Sep 14 08:06:45.287315 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:06:45.287292    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a10822_c5f6_42c8_bda9_b60fac1a8237.slice/crio-eb77e2fd356e58894ed0ac367f893a4f7635ea3eec7a69f1cd896aaea96013d3 WatchSource:0}: Error finding container eb77e2fd356e58894ed0ac367f893a4f7635ea3eec7a69f1cd896aaea96013d3: Status 404 returned error can't find the container with id eb77e2fd356e58894ed0ac367f893a4f7635ea3eec7a69f1cd896aaea96013d3
Sep 14 08:06:46.243776 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.243714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd" event={"ID":"04a10822-c5f6-42c8-bda9-b60fac1a8237","Type":"ContainerStarted","Data":"edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea"}
Sep 14 08:06:46.244244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.243803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd" event={"ID":"04a10822-c5f6-42c8-bda9-b60fac1a8237","Type":"ContainerStarted","Data":"eb77e2fd356e58894ed0ac367f893a4f7635ea3eec7a69f1cd896aaea96013d3"}
Sep 14 08:06:46.244244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.243833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:46.245505 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.245478    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" event={"ID":"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0","Type":"ContainerStarted","Data":"2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660"}
Sep 14 08:06:46.245650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.245525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" event={"ID":"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0","Type":"ContainerStarted","Data":"fbd8273b5e5b87675e5e05b5f87c6a5fc9861e27eadab2166e1cd7b9c30ebbf4"}
Sep 14 08:06:46.245650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.245554    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:46.247245 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.247221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl" event={"ID":"d19eea12-9ca8-40ea-b3ba-a916d0e9891a","Type":"ContainerStarted","Data":"7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677"}
Sep 14 08:06:46.247332 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.247263    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl" event={"ID":"d19eea12-9ca8-40ea-b3ba-a916d0e9891a","Type":"ContainerStarted","Data":"5c5a1b44e5c3a65f4ab6b03e9e6397ec3e00a9b353cec8aa17e1204fc78e1242"}
Sep 14 08:06:46.247456 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.247442    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:06:46.266896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.266853    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd" podStartSLOduration=2.266839525 podStartE2EDuration="2.266839525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:46.265680611 +0000 UTC m=+1061.973694752" watchObservedRunningTime="2026-09-14 08:06:46.266839525 +0000 UTC m=+1061.974853661"
Sep 14 08:06:46.286267 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.286222    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl" podStartSLOduration=2.2862052889999998 podStartE2EDuration="2.286205289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:46.285502525 +0000 UTC m=+1061.993516665" watchObservedRunningTime="2026-09-14 08:06:46.286205289 +0000 UTC m=+1061.994219429"
Sep 14 08:06:46.305526 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:46.305486    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" podStartSLOduration=1.946678816 podStartE2EDuration="2.305472259s" totalImagesPullingTime="358.793443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:46.304387192 +0000 UTC m=+1062.012401343" watchObservedRunningTime="2026-09-14 08:06:46.305472259 +0000 UTC m=+1062.013486399"
Sep 14 08:06:52.260166 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:52.260139    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:06:57.257957 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:57.257708    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:06:57.259019 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:06:57.258991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:07:02.342014 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:02.341976    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerID="b1af8d49f3a293f76554ecda9e6bea0fb18efbf25807f5811c579c69d098747f" exitCode=0
Sep 14 08:07:02.342528 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:02.342023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" event={"ID":"d5deb6c6-62c7-4936-b94d-25b5e92e36f7","Type":"ContainerDied","Data":"b1af8d49f3a293f76554ecda9e6bea0fb18efbf25807f5811c579c69d098747f"}
Sep 14 08:07:02.342528 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:02.342414    2563 scope.go:117] "RemoveContainer" containerID="b1af8d49f3a293f76554ecda9e6bea0fb18efbf25807f5811c579c69d098747f"
Sep 14 08:07:02.343684 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:02.343647    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:02.343799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:02.343753    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:02.343861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:02.343801    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:03.353837 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:03.353796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" event={"ID":"d5deb6c6-62c7-4936-b94d-25b5e92e36f7","Type":"ContainerStarted","Data":"03d906278b46fb3b3a902e25fb355a8ddcbc257c29c5386e1dd25922d99af293"}
Sep 14 08:07:04.251748 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:04.251690    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:07:04.251958 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:04.251829    2563 patch_prober.go:28] interesting pod/e2e-explicit-239599-openshift-default-f466b4968-qzbcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 14 08:07:04.251958 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:04.251877    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 14 08:07:05.252422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:05.252383    2563 patch_prober.go:28] interesting pod/e2e-explicit-239599-openshift-default-f466b4968-qzbcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 14 08:07:05.253039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:05.253002    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 14 08:07:06.251827 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:06.251783    2563 patch_prober.go:28] interesting pod/e2e-explicit-239599-openshift-default-f466b4968-qzbcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 14 08:07:06.252014 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:06.251851    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 14 08:07:07.252013 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:07.251952    2563 patch_prober.go:28] interesting pod/e2e-explicit-239599-openshift-default-f466b4968-qzbcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 14 08:07:07.252635 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:07.252040    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 14 08:07:08.251676 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:08.251631    2563 patch_prober.go:28] interesting pod/e2e-explicit-239599-openshift-default-f466b4968-qzbcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 14 08:07:08.251901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:08.251698    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 14 08:07:09.255828 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:09.255788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:07:09.256293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:09.256075    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:07:09.256835 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:09.256818    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:07:28.110480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.110442    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"]
Sep 14 08:07:28.116634 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.116611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.118928 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.118901    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-58a5b128-serving-cert\""
Sep 14 08:07:28.122198 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.122176    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"]
Sep 14 08:07:28.142597 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.142573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-metrics-certs\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.142704 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.142608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.142704 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.142634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqqfj\" (UniqueName: \"kubernetes.io/projected/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-kube-api-access-gqqfj\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.142803 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.142707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-openshift-service-ca\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.244183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.244148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqqfj\" (UniqueName: \"kubernetes.io/projected/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-kube-api-access-gqqfj\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.244323 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.244206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-openshift-service-ca\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.244388 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.244363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-metrics-certs\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.244428 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.244401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.244546 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:28.244531    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-serving-cert: secret "maas-api-e2e-mt-58a5b128-serving-cert" not found
Sep 14 08:07:28.244611 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:28.244601    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls podName:ea688cb0-170e-4fb6-9339-5fd7ff846bd4 nodeName:}" failed. No retries permitted until 2026-09-14 08:07:28.744579941 +0000 UTC m=+1104.452594060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls") pod "maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" (UID: "ea688cb0-170e-4fb6-9339-5fd7ff846bd4") : secret "maas-api-e2e-mt-58a5b128-serving-cert" not found
Sep 14 08:07:28.244941 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.244918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-openshift-service-ca\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.246896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.246872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-metrics-certs\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.254042 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.254023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqqfj\" (UniqueName: \"kubernetes.io/projected/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-kube-api-access-gqqfj\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.683559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.683525    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"]
Sep 14 08:07:28.687242 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.687224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.689637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.689616    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-58a5b128-dockercfg-xg7bh\""
Sep 14 08:07:28.689761 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.689694    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-58a5b128\""
Sep 14 08:07:28.698299 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.698269    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"]
Sep 14 08:07:28.748648 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.748618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nk5\" (UniqueName: \"kubernetes.io/projected/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-kube-api-access-54nk5\") pod \"payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.748861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.748697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.748861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.748780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.751777 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.751716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls\") pod \"maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:28.850061 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.850020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.850253 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.850087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nk5\" (UniqueName: \"kubernetes.io/projected/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-kube-api-access-54nk5\") pod \"payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.850712 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.850688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.857262 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.857232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nk5\" (UniqueName: \"kubernetes.io/projected/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-kube-api-access-54nk5\") pod \"payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:28.998944 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:28.998873    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:29.028899 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.028871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:29.278384 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.275585    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"]
Sep 14 08:07:29.278798 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:07:29.278663    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83276c2_f2f8_4d18_b3ce_65db67b4d0db.slice/crio-e7f7bc7139a17ae17a25b1cf91e963358f71066a0154846fbbb97a74e38cc135 WatchSource:0}: Error finding container e7f7bc7139a17ae17a25b1cf91e963358f71066a0154846fbbb97a74e38cc135: Status 404 returned error can't find the container with id e7f7bc7139a17ae17a25b1cf91e963358f71066a0154846fbbb97a74e38cc135
Sep 14 08:07:29.310671 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.310647    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"]
Sep 14 08:07:29.321959 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:07:29.321927    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea688cb0_170e_4fb6_9339_5fd7ff846bd4.slice/crio-2c5a77d4d9d5826e7f9ffe671b265f7ed692d430a670c891c19ca02e89d62697 WatchSource:0}: Error finding container 2c5a77d4d9d5826e7f9ffe671b265f7ed692d430a670c891c19ca02e89d62697: Status 404 returned error can't find the container with id 2c5a77d4d9d5826e7f9ffe671b265f7ed692d430a670c891c19ca02e89d62697
Sep 14 08:07:29.516861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.516825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" event={"ID":"ea688cb0-170e-4fb6-9339-5fd7ff846bd4","Type":"ContainerStarted","Data":"2c5a77d4d9d5826e7f9ffe671b265f7ed692d430a670c891c19ca02e89d62697"}
Sep 14 08:07:29.518695 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.518658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd" event={"ID":"f83276c2-f2f8-4d18-b3ce-65db67b4d0db","Type":"ContainerStarted","Data":"8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b"}
Sep 14 08:07:29.518856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.518707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd" event={"ID":"f83276c2-f2f8-4d18-b3ce-65db67b4d0db","Type":"ContainerStarted","Data":"e7f7bc7139a17ae17a25b1cf91e963358f71066a0154846fbbb97a74e38cc135"}
Sep 14 08:07:29.518856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.518763    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:29.535047 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:29.534964    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd" podStartSLOduration=1.534950851 podStartE2EDuration="1.534950851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:29.533025059 +0000 UTC m=+1105.241039198" watchObservedRunningTime="2026-09-14 08:07:29.534950851 +0000 UTC m=+1105.242965017"
Sep 14 08:07:30.525912 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:30.525877    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" event={"ID":"ea688cb0-170e-4fb6-9339-5fd7ff846bd4","Type":"ContainerStarted","Data":"92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8"}
Sep 14 08:07:30.526273 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:30.526211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:30.545120 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:30.545073    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" podStartSLOduration=1.9488635730000001 podStartE2EDuration="2.54505857s" totalImagesPullingTime="596.194997ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:30.541418394 +0000 UTC m=+1106.249432536" watchObservedRunningTime="2026-09-14 08:07:30.54505857 +0000 UTC m=+1106.253072708"
Sep 14 08:07:36.536694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:36.536664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:40.528111 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:40.528082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:43.496438 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.496400    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"]
Sep 14 08:07:43.496918 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.496647    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" podUID="05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" containerName="maas-api" containerID="cri-o://2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660" gracePeriod=30
Sep 14 08:07:43.541298 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.541264    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"]
Sep 14 08:07:43.541551 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.541523    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd" podUID="04a10822-c5f6-42c8-bda9-b60fac1a8237" containerName="payload-processing" containerID="cri-o://edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea" gracePeriod=30
Sep 14 08:07:43.549469 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.549107    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"]
Sep 14 08:07:43.549469 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.549327    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl" podUID="d19eea12-9ca8-40ea-b3ba-a916d0e9891a" containerName="payload-pre-processing" containerID="cri-o://7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677" gracePeriod=30
Sep 14 08:07:43.859304 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.859279    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:07:43.862395 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.862375    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:07:43.865831 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:43.865811    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:07:44.005119 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005088    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdsg2\" (UniqueName: \"kubernetes.io/projected/04a10822-c5f6-42c8-bda9-b60fac1a8237-kube-api-access-kdsg2\") pod \"04a10822-c5f6-42c8-bda9-b60fac1a8237\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") "
Sep 14 08:07:44.005315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005141    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fwjs\" (UniqueName: \"kubernetes.io/projected/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-kube-api-access-4fwjs\") pod \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") "
Sep 14 08:07:44.005315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005167    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-metrics-certs\") pod \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") "
Sep 14 08:07:44.005315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005198    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-plugins-config-volume\") pod \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\" (UID: \"d19eea12-9ca8-40ea-b3ba-a916d0e9891a\") "
Sep 14 08:07:44.005315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005225    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls\") pod \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") "
Sep 14 08:07:44.005315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005309    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpdq4\" (UniqueName: \"kubernetes.io/projected/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-kube-api-access-wpdq4\") pod \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") "
Sep 14 08:07:44.005607 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a10822-c5f6-42c8-bda9-b60fac1a8237-plugins-config-volume\") pod \"04a10822-c5f6-42c8-bda9-b60fac1a8237\" (UID: \"04a10822-c5f6-42c8-bda9-b60fac1a8237\") "
Sep 14 08:07:44.005811 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005780    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d19eea12-9ca8-40ea-b3ba-a916d0e9891a" (UID: "d19eea12-9ca8-40ea-b3ba-a916d0e9891a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:44.005906 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005838    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04a10822-c5f6-42c8-bda9-b60fac1a8237-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04a10822-c5f6-42c8-bda9-b60fac1a8237" (UID: "04a10822-c5f6-42c8-bda9-b60fac1a8237"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:44.005906 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.005887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-openshift-service-ca\") pod \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\" (UID: \"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0\") "
Sep 14 08:07:44.006299 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.006263    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" (UID: "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:44.006508 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.006488    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.006575 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.006513    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04a10822-c5f6-42c8-bda9-b60fac1a8237-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.006575 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.006529    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.007476 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.007456    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" (UID: "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:44.007554 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.007523    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-kube-api-access-wpdq4" (OuterVolumeSpecName: "kube-api-access-wpdq4") pod "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" (UID: "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0"). InnerVolumeSpecName "kube-api-access-wpdq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:44.007593 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.007552    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-kube-api-access-4fwjs" (OuterVolumeSpecName: "kube-api-access-4fwjs") pod "d19eea12-9ca8-40ea-b3ba-a916d0e9891a" (UID: "d19eea12-9ca8-40ea-b3ba-a916d0e9891a"). InnerVolumeSpecName "kube-api-access-4fwjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:44.007735 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.007696    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" (UID: "05101f4b-a8dc-48c9-823c-5e6a9ddab7a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:44.007735 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.007707    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a10822-c5f6-42c8-bda9-b60fac1a8237-kube-api-access-kdsg2" (OuterVolumeSpecName: "kube-api-access-kdsg2") pod "04a10822-c5f6-42c8-bda9-b60fac1a8237" (UID: "04a10822-c5f6-42c8-bda9-b60fac1a8237"). InnerVolumeSpecName "kube-api-access-kdsg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:44.107616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.107543    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdsg2\" (UniqueName: \"kubernetes.io/projected/04a10822-c5f6-42c8-bda9-b60fac1a8237-kube-api-access-kdsg2\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.107616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.107568    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fwjs\" (UniqueName: \"kubernetes.io/projected/d19eea12-9ca8-40ea-b3ba-a916d0e9891a-kube-api-access-4fwjs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.107616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.107579    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.107616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.107588    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.107616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.107598    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpdq4\" (UniqueName: \"kubernetes.io/projected/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0-kube-api-access-wpdq4\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:44.626751 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.626706    2563 generic.go:356] "Generic (PLEG): container finished" podID="04a10822-c5f6-42c8-bda9-b60fac1a8237" containerID="edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea" exitCode=0
Sep 14 08:07:44.627250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.626782    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"
Sep 14 08:07:44.627250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.626795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd" event={"ID":"04a10822-c5f6-42c8-bda9-b60fac1a8237","Type":"ContainerDied","Data":"edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea"}
Sep 14 08:07:44.627250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.626880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd" event={"ID":"04a10822-c5f6-42c8-bda9-b60fac1a8237","Type":"ContainerDied","Data":"eb77e2fd356e58894ed0ac367f893a4f7635ea3eec7a69f1cd896aaea96013d3"}
Sep 14 08:07:44.627250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.626909    2563 scope.go:117] "RemoveContainer" containerID="edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea"
Sep 14 08:07:44.628293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.628175    2563 generic.go:356] "Generic (PLEG): container finished" podID="05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" containerID="2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660" exitCode=0
Sep 14 08:07:44.628293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.628258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" event={"ID":"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0","Type":"ContainerDied","Data":"2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660"}
Sep 14 08:07:44.628422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.628299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk" event={"ID":"05101f4b-a8dc-48c9-823c-5e6a9ddab7a0","Type":"ContainerDied","Data":"fbd8273b5e5b87675e5e05b5f87c6a5fc9861e27eadab2166e1cd7b9c30ebbf4"}
Sep 14 08:07:44.628422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.628262    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"
Sep 14 08:07:44.629709 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.629687    2563 generic.go:356] "Generic (PLEG): container finished" podID="d19eea12-9ca8-40ea-b3ba-a916d0e9891a" containerID="7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677" exitCode=0
Sep 14 08:07:44.629850 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.629752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl" event={"ID":"d19eea12-9ca8-40ea-b3ba-a916d0e9891a","Type":"ContainerDied","Data":"7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677"}
Sep 14 08:07:44.629850 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.629799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl" event={"ID":"d19eea12-9ca8-40ea-b3ba-a916d0e9891a","Type":"ContainerDied","Data":"5c5a1b44e5c3a65f4ab6b03e9e6397ec3e00a9b353cec8aa17e1204fc78e1242"}
Sep 14 08:07:44.629850 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.629763    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"
Sep 14 08:07:44.637329 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.637291    2563 scope.go:117] "RemoveContainer" containerID="edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea"
Sep 14 08:07:44.637599 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:44.637576    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea\": container with ID starting with edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea not found: ID does not exist" containerID="edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea"
Sep 14 08:07:44.637658 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.637610    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea"} err="failed to get container status \"edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea\": rpc error: code = NotFound desc = could not find container \"edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea\": container with ID starting with edd4e4a920685fd2c43a7d20d1b692966aa201260cf778a16aeb662267c68dea not found: ID does not exist"
Sep 14 08:07:44.637658 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.637634    2563 scope.go:117] "RemoveContainer" containerID="2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660"
Sep 14 08:07:44.646650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.646634    2563 scope.go:117] "RemoveContainer" containerID="2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660"
Sep 14 08:07:44.646939 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:44.646922    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660\": container with ID starting with 2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660 not found: ID does not exist" containerID="2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660"
Sep 14 08:07:44.646995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.646946    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660"} err="failed to get container status \"2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660\": rpc error: code = NotFound desc = could not find container \"2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660\": container with ID starting with 2922348938af6b22b363f2f01bb04a1c843cef63013f6a30d3a711456b709660 not found: ID does not exist"
Sep 14 08:07:44.646995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.646961    2563 scope.go:117] "RemoveContainer" containerID="7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677"
Sep 14 08:07:44.654099 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.654077    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"]
Sep 14 08:07:44.655539 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.655471    2563 scope.go:117] "RemoveContainer" containerID="7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677"
Sep 14 08:07:44.655886 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:44.655785    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677\": container with ID starting with 7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677 not found: ID does not exist" containerID="7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677"
Sep 14 08:07:44.655886 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.655818    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677"} err="failed to get container status \"7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677\": rpc error: code = NotFound desc = could not find container \"7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677\": container with ID starting with 7c9ff7cb7dff2461cd797f0c2be55dd7c437318edd0930d48f278c155261c677 not found: ID does not exist"
Sep 14 08:07:44.658890 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.658869    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-239599-bf8fc869c-m96vd"]
Sep 14 08:07:44.668597 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.668577    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"]
Sep 14 08:07:44.672658 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.672637    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-239599-c8f576494-rjftl"]
Sep 14 08:07:44.682832 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.682810    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"]
Sep 14 08:07:44.688882 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.688863    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-239599-8dbd6dbbc-jg6zk"]
Sep 14 08:07:44.790217 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.790150    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a10822-c5f6-42c8-bda9-b60fac1a8237" path="/var/lib/kubelet/pods/04a10822-c5f6-42c8-bda9-b60fac1a8237/volumes"
Sep 14 08:07:44.790554 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.790541    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" path="/var/lib/kubelet/pods/05101f4b-a8dc-48c9-823c-5e6a9ddab7a0/volumes"
Sep 14 08:07:44.790900 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:44.790886    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19eea12-9ca8-40ea-b3ba-a916d0e9891a" path="/var/lib/kubelet/pods/d19eea12-9ca8-40ea-b3ba-a916d0e9891a/volumes"
Sep 14 08:07:46.728193 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728159    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"]
Sep 14 08:07:46.728596 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728584    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" containerName="maas-api"
Sep 14 08:07:46.728596 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728597    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" containerName="maas-api"
Sep 14 08:07:46.728675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728621    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d19eea12-9ca8-40ea-b3ba-a916d0e9891a" containerName="payload-pre-processing"
Sep 14 08:07:46.728675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728626    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19eea12-9ca8-40ea-b3ba-a916d0e9891a" containerName="payload-pre-processing"
Sep 14 08:07:46.728675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728633    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a10822-c5f6-42c8-bda9-b60fac1a8237" containerName="payload-processing"
Sep 14 08:07:46.728675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728639    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a10822-c5f6-42c8-bda9-b60fac1a8237" containerName="payload-processing"
Sep 14 08:07:46.728846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728711    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d19eea12-9ca8-40ea-b3ba-a916d0e9891a" containerName="payload-pre-processing"
Sep 14 08:07:46.728846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728721    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05101f4b-a8dc-48c9-823c-5e6a9ddab7a0" containerName="maas-api"
Sep 14 08:07:46.728846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.728741    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a10822-c5f6-42c8-bda9-b60fac1a8237" containerName="payload-processing"
Sep 14 08:07:46.733488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.733469    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:46.735957 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.735932    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:07:46.738295 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.738249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"]
Sep 14 08:07:46.833465 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.833433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h\" (UID: \"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:46.934858 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.934823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h\" (UID: \"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:46.935488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:46.935465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h\" (UID: \"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:47.046427 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:47.046357    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:47.318404 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:47.318379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"]
Sep 14 08:07:47.320038 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:07:47.320010    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d596f1_fbfb_4b2c_b5f7_8145797fef6c.slice/crio-43b48e9f0995a65d43a79fbf293add081d49b787105c4fc5c38fbdcd0f26a659 WatchSource:0}: Error finding container 43b48e9f0995a65d43a79fbf293add081d49b787105c4fc5c38fbdcd0f26a659: Status 404 returned error can't find the container with id 43b48e9f0995a65d43a79fbf293add081d49b787105c4fc5c38fbdcd0f26a659
Sep 14 08:07:47.652822 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:47.652793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h" event={"ID":"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c","Type":"ContainerStarted","Data":"43b48e9f0995a65d43a79fbf293add081d49b787105c4fc5c38fbdcd0f26a659"}
Sep 14 08:07:48.659995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:48.659955    2563 generic.go:356] "Generic (PLEG): container finished" podID="d2d596f1-fbfb-4b2c-b5f7-8145797fef6c" containerID="9c523c034b1477e95fb1a8b58cbd3a17d4393eb8b678974ed3c954b49bebba61" exitCode=0
Sep 14 08:07:48.660368 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:48.660039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h" event={"ID":"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c","Type":"ContainerDied","Data":"9c523c034b1477e95fb1a8b58cbd3a17d4393eb8b678974ed3c954b49bebba61"}
Sep 14 08:07:49.802882 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:49.802859    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:49.962995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:49.962964    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c-maas-api-service-ca\") pod \"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c\" (UID: \"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c\") "
Sep 14 08:07:49.963354 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:49.963323    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2d596f1-fbfb-4b2c-b5f7-8145797fef6c" (UID: "d2d596f1-fbfb-4b2c-b5f7-8145797fef6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:50.064631 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:50.064599    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:50.570288 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:50.570243    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"]
Sep 14 08:07:50.570464 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:50.570445    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" containerID="cri-o://03d906278b46fb3b3a902e25fb355a8ddcbc257c29c5386e1dd25922d99af293" gracePeriod=30
Sep 14 08:07:50.674306 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:50.674269    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"
Sep 14 08:07:50.674306 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:50.674295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h" event={"ID":"d2d596f1-fbfb-4b2c-b5f7-8145797fef6c","Type":"ContainerDied","Data":"43b48e9f0995a65d43a79fbf293add081d49b787105c4fc5c38fbdcd0f26a659"}
Sep 14 08:07:50.674552 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:50.674341    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43b48e9f0995a65d43a79fbf293add081d49b787105c4fc5c38fbdcd0f26a659"
Sep 14 08:07:51.493147 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.493109    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"]
Sep 14 08:07:51.493781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.493761    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d596f1-fbfb-4b2c-b5f7-8145797fef6c" containerName="revoke-keys"
Sep 14 08:07:51.493781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.493781    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d596f1-fbfb-4b2c-b5f7-8145797fef6c" containerName="revoke-keys"
Sep 14 08:07:51.493949 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.493883    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d596f1-fbfb-4b2c-b5f7-8145797fef6c" containerName="revoke-keys"
Sep 14 08:07:51.497340 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.497321    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.499689 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.499661    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-1af772-openshift-default-dockercfg-446w9\""
Sep 14 08:07:51.509645 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.509617    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"]
Sep 14 08:07:51.680925 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.680887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-envoy\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.680933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-token\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.680958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-credential-socket\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.681017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istiod-ca-cert\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.681055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-podinfo\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.681073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-certs\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681286 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.681094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhr7v\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-kube-api-access-jhr7v\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681286 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.681116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-data\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.681286 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.681196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-socket\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-envoy\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-token\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782422 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-credential-socket\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istiod-ca-cert\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-podinfo\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-certs\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhr7v\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-kube-api-access-jhr7v\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-data\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.782699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-socket\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.783037 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-credential-socket\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.783037 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.782985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-data\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.783151 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.783125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-certs\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.783211 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.783148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-socket\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.783265 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.783245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istiod-ca-cert\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.785368 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.785345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-envoy\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.785475 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.785423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-podinfo\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.791849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.791826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhr7v\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-kube-api-access-jhr7v\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.792003 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.791984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-token\") pod \"e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") " pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:51.810740 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:51.810707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:52.100397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.100370    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"]
Sep 14 08:07:52.101535 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:07:52.101500    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39632f7a_0269_4e67_a239_1a8eb55dd0b8.slice/crio-57681d1fade6ae117290a9f72ca513605e9d22b78d5644625ad95ebbc42fd082 WatchSource:0}: Error finding container 57681d1fade6ae117290a9f72ca513605e9d22b78d5644625ad95ebbc42fd082: Status 404 returned error can't find the container with id 57681d1fade6ae117290a9f72ca513605e9d22b78d5644625ad95ebbc42fd082
Sep 14 08:07:52.103932 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.103897    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:52.104001 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.103981    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:52.104037 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.104014    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:52.692045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.691986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" event={"ID":"39632f7a-0269-4e67-a239-1a8eb55dd0b8","Type":"ContainerStarted","Data":"0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db"}
Sep 14 08:07:52.692506 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.692052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" event={"ID":"39632f7a-0269-4e67-a239-1a8eb55dd0b8","Type":"ContainerStarted","Data":"57681d1fade6ae117290a9f72ca513605e9d22b78d5644625ad95ebbc42fd082"}
Sep 14 08:07:52.712454 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.712365    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" podStartSLOduration=1.712345489 podStartE2EDuration="1.712345489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:52.71146838 +0000 UTC m=+1128.419482530" watchObservedRunningTime="2026-09-14 08:07:52.712345489 +0000 UTC m=+1128.420359630"
Sep 14 08:07:52.810806 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:52.810778    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:53.816400 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:53.816371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:54.258862 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:54.258823    2563 patch_prober.go:28] interesting pod/e2e-explicit-239599-openshift-default-f466b4968-qzbcr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:07:54.259030 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:54.258891    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:07:54.703135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:54.703102    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:54.704176 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:54.704157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:07:55.712034 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.712001    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerID="03d906278b46fb3b3a902e25fb355a8ddcbc257c29c5386e1dd25922d99af293" exitCode=0
Sep 14 08:07:55.712409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.712076    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" event={"ID":"d5deb6c6-62c7-4936-b94d-25b5e92e36f7","Type":"ContainerDied","Data":"03d906278b46fb3b3a902e25fb355a8ddcbc257c29c5386e1dd25922d99af293"}
Sep 14 08:07:55.712409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.712154    2563 scope.go:117] "RemoveContainer" containerID="b1af8d49f3a293f76554ecda9e6bea0fb18efbf25807f5811c579c69d098747f"
Sep 14 08:07:55.825780 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.825758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:07:55.924903 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.924828    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbzsg\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-kube-api-access-qbzsg\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.924903 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.924878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-credential-socket\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.924909    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-data\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.924937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-token\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.924983    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-socket\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-envoy\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-certs\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925376 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925127    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-podinfo\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925376 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925168    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istiod-ca-cert\") pod \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\" (UID: \"d5deb6c6-62c7-4936-b94d-25b5e92e36f7\") "
Sep 14 08:07:55.925376 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925246    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:55.925376 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925261    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:55.925376 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925279    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:55.925646 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925412    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:55.925646 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925580    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:55.925799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925649    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istiod-ca-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:55.925799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925666    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-credential-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:55.925799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925676    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-data\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:55.925799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925689    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:55.925799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.925699    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-workload-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:55.927177 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.927145    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-kube-api-access-qbzsg" (OuterVolumeSpecName: "kube-api-access-qbzsg") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "kube-api-access-qbzsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:55.927294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.927260    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:55.927358 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.927326    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:07:55.927416 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:55.927355    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5deb6c6-62c7-4936-b94d-25b5e92e36f7" (UID: "d5deb6c6-62c7-4936-b94d-25b5e92e36f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:56.026667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.026582    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-podinfo\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.026667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.026615    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbzsg\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-kube-api-access-qbzsg\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.026667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.026632    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.026667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.026644    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5deb6c6-62c7-4936-b94d-25b5e92e36f7-istio-envoy\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.036297 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.036263    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"]
Sep 14 08:07:56.036496 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.036459    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" podUID="ea688cb0-170e-4fb6-9339-5fd7ff846bd4" containerName="maas-api" containerID="cri-o://92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8" gracePeriod=30
Sep 14 08:07:56.098340 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.098307    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"]
Sep 14 08:07:56.098562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.098520    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd" podUID="f83276c2-f2f8-4d18-b3ce-65db67b4d0db" containerName="payload-pre-processing" containerID="cri-o://8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b" gracePeriod=30
Sep 14 08:07:56.246864 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.246783    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"]
Sep 14 08:07:56.250055 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.250029    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-58a5b128-fjw7h"]
Sep 14 08:07:56.357504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.357480    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:56.366590 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.366567    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:56.532268 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54nk5\" (UniqueName: \"kubernetes.io/projected/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-kube-api-access-54nk5\") pod \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") "
Sep 14 08:07:56.532268 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-openshift-service-ca\") pod \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") "
Sep 14 08:07:56.532453 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532422    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-metrics-certs\") pod \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") "
Sep 14 08:07:56.532505 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532463    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls\") pod \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") "
Sep 14 08:07:56.532561 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532510    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-plugins-config-volume\") pod \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\" (UID: \"f83276c2-f2f8-4d18-b3ce-65db67b4d0db\") "
Sep 14 08:07:56.532561 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532544    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea688cb0-170e-4fb6-9339-5fd7ff846bd4" (UID: "ea688cb0-170e-4fb6-9339-5fd7ff846bd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:56.532665 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqqfj\" (UniqueName: \"kubernetes.io/projected/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-kube-api-access-gqqfj\") pod \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\" (UID: \"ea688cb0-170e-4fb6-9339-5fd7ff846bd4\") "
Sep 14 08:07:56.533022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.532993    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f83276c2-f2f8-4d18-b3ce-65db67b4d0db" (UID: "f83276c2-f2f8-4d18-b3ce-65db67b4d0db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:56.533022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.533015    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.534559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.534539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea688cb0-170e-4fb6-9339-5fd7ff846bd4" (UID: "ea688cb0-170e-4fb6-9339-5fd7ff846bd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:56.534559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.534553    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-kube-api-access-gqqfj" (OuterVolumeSpecName: "kube-api-access-gqqfj") pod "ea688cb0-170e-4fb6-9339-5fd7ff846bd4" (UID: "ea688cb0-170e-4fb6-9339-5fd7ff846bd4"). InnerVolumeSpecName "kube-api-access-gqqfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:56.534865 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.534840    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea688cb0-170e-4fb6-9339-5fd7ff846bd4" (UID: "ea688cb0-170e-4fb6-9339-5fd7ff846bd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:56.534865 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.534850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-kube-api-access-54nk5" (OuterVolumeSpecName: "kube-api-access-54nk5") pod "f83276c2-f2f8-4d18-b3ce-65db67b4d0db" (UID: "f83276c2-f2f8-4d18-b3ce-65db67b4d0db"). InnerVolumeSpecName "kube-api-access-54nk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:56.633968 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.633932    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.633968 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.633967    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.633968 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.633976    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.634132 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.633987    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqqfj\" (UniqueName: \"kubernetes.io/projected/ea688cb0-170e-4fb6-9339-5fd7ff846bd4-kube-api-access-gqqfj\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.634132 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.633996    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54nk5\" (UniqueName: \"kubernetes.io/projected/f83276c2-f2f8-4d18-b3ce-65db67b4d0db-kube-api-access-54nk5\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:07:56.719506 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.719474    2563 generic.go:356] "Generic (PLEG): container finished" podID="ea688cb0-170e-4fb6-9339-5fd7ff846bd4" containerID="92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8" exitCode=0
Sep 14 08:07:56.719973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.719515    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"
Sep 14 08:07:56.719973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.719565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" event={"ID":"ea688cb0-170e-4fb6-9339-5fd7ff846bd4","Type":"ContainerDied","Data":"92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8"}
Sep 14 08:07:56.719973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.719616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj" event={"ID":"ea688cb0-170e-4fb6-9339-5fd7ff846bd4","Type":"ContainerDied","Data":"2c5a77d4d9d5826e7f9ffe671b265f7ed692d430a670c891c19ca02e89d62697"}
Sep 14 08:07:56.719973 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.719632    2563 scope.go:117] "RemoveContainer" containerID="92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8"
Sep 14 08:07:56.721209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.721187    2563 generic.go:356] "Generic (PLEG): container finished" podID="f83276c2-f2f8-4d18-b3ce-65db67b4d0db" containerID="8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b" exitCode=0
Sep 14 08:07:56.721354 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.721230    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"
Sep 14 08:07:56.721354 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.721273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd" event={"ID":"f83276c2-f2f8-4d18-b3ce-65db67b4d0db","Type":"ContainerDied","Data":"8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b"}
Sep 14 08:07:56.721354 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.721322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd" event={"ID":"f83276c2-f2f8-4d18-b3ce-65db67b4d0db","Type":"ContainerDied","Data":"e7f7bc7139a17ae17a25b1cf91e963358f71066a0154846fbbb97a74e38cc135"}
Sep 14 08:07:56.723130 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.723103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr" event={"ID":"d5deb6c6-62c7-4936-b94d-25b5e92e36f7","Type":"ContainerDied","Data":"ee3d74a69a279b31eec651c3e73589fb4cb2efaaeda529975bc6f9f9287d3a31"}
Sep 14 08:07:56.723130 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.723121    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"
Sep 14 08:07:56.731084 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.731065    2563 scope.go:117] "RemoveContainer" containerID="92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8"
Sep 14 08:07:56.731427 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:56.731404    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8\": container with ID starting with 92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8 not found: ID does not exist" containerID="92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8"
Sep 14 08:07:56.731521 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.731432    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8"} err="failed to get container status \"92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8\": rpc error: code = NotFound desc = could not find container \"92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8\": container with ID starting with 92294c92258ec031250c9de05074818c82e8747d6a83f33e942ff3ead36282c8 not found: ID does not exist"
Sep 14 08:07:56.731521 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.731448    2563 scope.go:117] "RemoveContainer" containerID="8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b"
Sep 14 08:07:56.740567 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.740551    2563 scope.go:117] "RemoveContainer" containerID="8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b"
Sep 14 08:07:56.740852 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:07:56.740834    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b\": container with ID starting with 8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b not found: ID does not exist" containerID="8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b"
Sep 14 08:07:56.740946 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.740855    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b"} err="failed to get container status \"8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b\": rpc error: code = NotFound desc = could not find container \"8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b\": container with ID starting with 8894818d1a21713ff408f1e1ecb944709b059d0714d0466ee5b2c6aee2e8362b not found: ID does not exist"
Sep 14 08:07:56.740946 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.740869    2563 scope.go:117] "RemoveContainer" containerID="03d906278b46fb3b3a902e25fb355a8ddcbc257c29c5386e1dd25922d99af293"
Sep 14 08:07:56.748205 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.748185    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"]
Sep 14 08:07:56.752666 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.752646    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58a5b128-d44bdcdc5-jslnj"]
Sep 14 08:07:56.763397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.763378    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"]
Sep 14 08:07:56.767237 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.767218    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-239599-openshift-default-f466b4968-qzbcr"]
Sep 14 08:07:56.776826 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.776806    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"]
Sep 14 08:07:56.778537 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.778520    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58a5b128-696b696fb6-nccxd"]
Sep 14 08:07:56.789505 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.789446    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2d596f1-fbfb-4b2c-b5f7-8145797fef6c" path="/var/lib/kubelet/pods/d2d596f1-fbfb-4b2c-b5f7-8145797fef6c/volumes"
Sep 14 08:07:56.789977 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.789957    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" path="/var/lib/kubelet/pods/d5deb6c6-62c7-4936-b94d-25b5e92e36f7/volumes"
Sep 14 08:07:56.790384 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.790366    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea688cb0-170e-4fb6-9339-5fd7ff846bd4" path="/var/lib/kubelet/pods/ea688cb0-170e-4fb6-9339-5fd7ff846bd4/volumes"
Sep 14 08:07:56.791023 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:07:56.791003    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83276c2-f2f8-4d18-b3ce-65db67b4d0db" path="/var/lib/kubelet/pods/f83276c2-f2f8-4d18-b3ce-65db67b4d0db/volumes"
Sep 14 08:08:16.384182 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384139    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"]
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384555    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83276c2-f2f8-4d18-b3ce-65db67b4d0db" containerName="payload-pre-processing"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384567    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83276c2-f2f8-4d18-b3ce-65db67b4d0db" containerName="payload-pre-processing"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384591    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384597    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384606    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea688cb0-170e-4fb6-9339-5fd7ff846bd4" containerName="maas-api"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384611    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea688cb0-170e-4fb6-9339-5fd7ff846bd4" containerName="maas-api"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384620    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy"
Sep 14 08:08:16.384624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384625    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy"
Sep 14 08:08:16.385001 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384686    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy"
Sep 14 08:08:16.385001 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384696    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5deb6c6-62c7-4936-b94d-25b5e92e36f7" containerName="istio-proxy"
Sep 14 08:08:16.385001 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384703    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83276c2-f2f8-4d18-b3ce-65db67b4d0db" containerName="payload-pre-processing"
Sep 14 08:08:16.385001 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.384711    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea688cb0-170e-4fb6-9339-5fd7ff846bd4" containerName="maas-api"
Sep 14 08:08:16.389836 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.389814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.392106 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.392085    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-0b8346-openshift-default-dockercfg-f5w72\""
Sep 14 08:08:16.402239 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.402214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"]
Sep 14 08:08:16.522276 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-token\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-envoy\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbh45\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-kube-api-access-lbh45\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-data\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istiod-ca-cert\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-certs\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-podinfo\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522525    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-credential-socket\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.522691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.522615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-socket\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623229 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istiod-ca-cert\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623229 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-certs\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-podinfo\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-credential-socket\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-socket\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-token\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-envoy\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbh45\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-kube-api-access-lbh45\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-data\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623828 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-credential-socket\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623828 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-socket\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.623937 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-data\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.624002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.623990    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-certs\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.624061 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.624048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istiod-ca-cert\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.625877 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.625848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-envoy\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.626164 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.626142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-podinfo\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.630835 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.630812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-token\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.631207 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.631187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbh45\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-kube-api-access-lbh45\") pod \"e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") " pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.702102 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.702070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:16.998421 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:16.998395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"]
Sep 14 08:08:17.000273 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:17.000235    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d7fb36_40c1_41e3_94e2_84ac253f0c76.slice/crio-4ca18f35e903d74e80ec26eb1891647afc5203dc9c19a00281ee9b3b4dda7f68 WatchSource:0}: Error finding container 4ca18f35e903d74e80ec26eb1891647afc5203dc9c19a00281ee9b3b4dda7f68: Status 404 returned error can't find the container with id 4ca18f35e903d74e80ec26eb1891647afc5203dc9c19a00281ee9b3b4dda7f68
Sep 14 08:08:17.003294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:17.003213    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:17.003385 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:17.003298    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:17.003385 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:17.003345    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:17.847178 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:17.847141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" event={"ID":"07d7fb36-40c1-41e3-94e2-84ac253f0c76","Type":"ContainerStarted","Data":"f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"}
Sep 14 08:08:17.847551 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:17.847195    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" event={"ID":"07d7fb36-40c1-41e3-94e2-84ac253f0c76","Type":"ContainerStarted","Data":"4ca18f35e903d74e80ec26eb1891647afc5203dc9c19a00281ee9b3b4dda7f68"}
Sep 14 08:08:17.867473 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:17.867421    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podStartSLOduration=1.867406149 podStartE2EDuration="1.867406149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:17.866311321 +0000 UTC m=+1153.574325467" watchObservedRunningTime="2026-09-14 08:08:17.867406149 +0000 UTC m=+1153.575420288"
Sep 14 08:08:18.702467 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:18.702431    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:18.707620 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:18.707593    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:18.853268 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:18.853229    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:18.854299 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:18.854271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:22.704852 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.704813    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"]
Sep 14 08:08:22.709654 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.709630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.712235 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.712216    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-0b8346-serving-cert\""
Sep 14 08:08:22.712335 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.712240    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:08:22.712395 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.712336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:08:22.713196 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.713178    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:08:22.713295 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.713236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:08:22.718648 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.718625    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"]
Sep 14 08:08:22.785043 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.785013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvhcs\" (UniqueName: \"kubernetes.io/projected/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-kube-api-access-fvhcs\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.785220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.785058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.785220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.785108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.785343 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.785225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.885929 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.885898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.886087 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.885972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvhcs\" (UniqueName: \"kubernetes.io/projected/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-kube-api-access-fvhcs\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.886087 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.886002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.886087 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.886024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.886200 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:08:22.886141    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-serving-cert: secret "maas-api-e2e-shared-a-w2-0b8346-serving-cert" not found
Sep 14 08:08:22.886200 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:08:22.886198    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls podName:3a3dcf97-eb10-4dc1-92a8-497fd83363ea nodeName:}" failed. No retries permitted until 2026-09-14 08:08:23.386181083 +0000 UTC m=+1159.094195200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls") pod "maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" (UID: "3a3dcf97-eb10-4dc1-92a8-497fd83363ea") : secret "maas-api-e2e-shared-a-w2-0b8346-serving-cert" not found
Sep 14 08:08:22.886668 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.886646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.888371 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.888349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:22.894061 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:22.894036    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvhcs\" (UniqueName: \"kubernetes.io/projected/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-kube-api-access-fvhcs\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:23.293360 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.293331    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"]
Sep 14 08:08:23.297120 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.297105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.299683 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.299662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-0b8346-dockercfg-25xnq\""
Sep 14 08:08:23.299818 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.299662    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-0b8346\""
Sep 14 08:08:23.306754 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.306715    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"]
Sep 14 08:08:23.335269 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.335246    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"]
Sep 14 08:08:23.345030 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.345000    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.350429 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.350402    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"]
Sep 14 08:08:23.392372 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.392328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ea5649-56da-4ebf-9515-da506c65cd74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.392523 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.392424    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:23.392586 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.392514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w77t\" (UniqueName: \"kubernetes.io/projected/06ea5649-56da-4ebf-9515-da506c65cd74-kube-api-access-2w77t\") pod \"payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.394933 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.394908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:23.493618 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.493583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ea5649-56da-4ebf-9515-da506c65cd74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.493820 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.493662    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w77t\" (UniqueName: \"kubernetes.io/projected/06ea5649-56da-4ebf-9515-da506c65cd74-kube-api-access-2w77t\") pod \"payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.493820 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.493695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qlrr\" (UniqueName: \"kubernetes.io/projected/669ebfb4-dc18-4ef5-99c4-0d626661bff7-kube-api-access-6qlrr\") pod \"payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.493820 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.493715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/669ebfb4-dc18-4ef5-99c4-0d626661bff7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.494394 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.494372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ea5649-56da-4ebf-9515-da506c65cd74-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.501274 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.501252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w77t\" (UniqueName: \"kubernetes.io/projected/06ea5649-56da-4ebf-9515-da506c65cd74-kube-api-access-2w77t\") pod \"payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.595379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.595312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qlrr\" (UniqueName: \"kubernetes.io/projected/669ebfb4-dc18-4ef5-99c4-0d626661bff7-kube-api-access-6qlrr\") pod \"payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.595379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.595348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/669ebfb4-dc18-4ef5-99c4-0d626661bff7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.596066 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.596048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/669ebfb4-dc18-4ef5-99c4-0d626661bff7-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.602262 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.602241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qlrr\" (UniqueName: \"kubernetes.io/projected/669ebfb4-dc18-4ef5-99c4-0d626661bff7-kube-api-access-6qlrr\") pod \"payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.608112 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.608086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:23.624705 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.624687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:23.660101 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.659957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:23.883324 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.883302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"]
Sep 14 08:08:23.884544 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:23.884516    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06ea5649_56da_4ebf_9515_da506c65cd74.slice/crio-223dc8bdbab1e798bd538f2ac940942104ea44f724707f7f44558da2a43399a5 WatchSource:0}: Error finding container 223dc8bdbab1e798bd538f2ac940942104ea44f724707f7f44558da2a43399a5: Status 404 returned error can't find the container with id 223dc8bdbab1e798bd538f2ac940942104ea44f724707f7f44558da2a43399a5
Sep 14 08:08:23.909496 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.909470    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"]
Sep 14 08:08:23.910458 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:23.910439    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a3dcf97_eb10_4dc1_92a8_497fd83363ea.slice/crio-dfc6976b6fd945d7d2881455342f5963eb18f061d9891e7ec8869bf64640a930 WatchSource:0}: Error finding container dfc6976b6fd945d7d2881455342f5963eb18f061d9891e7ec8869bf64640a930: Status 404 returned error can't find the container with id dfc6976b6fd945d7d2881455342f5963eb18f061d9891e7ec8869bf64640a930
Sep 14 08:08:23.946931 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:23.946909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"]
Sep 14 08:08:23.949569 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:23.949525    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod669ebfb4_dc18_4ef5_99c4_0d626661bff7.slice/crio-55f46889335aa257f5b486238f6edd7e386598fc0524a5f797991357bc5cb8e3 WatchSource:0}: Error finding container 55f46889335aa257f5b486238f6edd7e386598fc0524a5f797991357bc5cb8e3: Status 404 returned error can't find the container with id 55f46889335aa257f5b486238f6edd7e386598fc0524a5f797991357bc5cb8e3
Sep 14 08:08:24.890716 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.890676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v" event={"ID":"06ea5649-56da-4ebf-9515-da506c65cd74","Type":"ContainerStarted","Data":"342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73"}
Sep 14 08:08:24.891135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.890757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v" event={"ID":"06ea5649-56da-4ebf-9515-da506c65cd74","Type":"ContainerStarted","Data":"223dc8bdbab1e798bd538f2ac940942104ea44f724707f7f44558da2a43399a5"}
Sep 14 08:08:24.891135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.890787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:24.892432 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.892406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72" event={"ID":"669ebfb4-dc18-4ef5-99c4-0d626661bff7","Type":"ContainerStarted","Data":"fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1"}
Sep 14 08:08:24.892549 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.892454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72" event={"ID":"669ebfb4-dc18-4ef5-99c4-0d626661bff7","Type":"ContainerStarted","Data":"55f46889335aa257f5b486238f6edd7e386598fc0524a5f797991357bc5cb8e3"}
Sep 14 08:08:24.892549 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.892483    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:24.894020 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.893996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" event={"ID":"3a3dcf97-eb10-4dc1-92a8-497fd83363ea","Type":"ContainerStarted","Data":"9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2"}
Sep 14 08:08:24.894103 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.894038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" event={"ID":"3a3dcf97-eb10-4dc1-92a8-497fd83363ea","Type":"ContainerStarted","Data":"dfc6976b6fd945d7d2881455342f5963eb18f061d9891e7ec8869bf64640a930"}
Sep 14 08:08:24.894103 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.894069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:24.907811 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.907768    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v" podStartSLOduration=1.90775514 podStartE2EDuration="1.90775514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:24.906538775 +0000 UTC m=+1160.614552916" watchObservedRunningTime="2026-09-14 08:08:24.90775514 +0000 UTC m=+1160.615769278"
Sep 14 08:08:24.921954 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.921921    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" podStartSLOduration=2.4490456480000002 podStartE2EDuration="2.921912893s" totalImagesPullingTime="472.867245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:24.921296828 +0000 UTC m=+1160.629310968" watchObservedRunningTime="2026-09-14 08:08:24.921912893 +0000 UTC m=+1160.629927032"
Sep 14 08:08:24.937395 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:24.937355    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72" podStartSLOduration=1.937342374 podStartE2EDuration="1.937342374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:24.934602932 +0000 UTC m=+1160.642617071" watchObservedRunningTime="2026-09-14 08:08:24.937342374 +0000 UTC m=+1160.645356516"
Sep 14 08:08:30.904997 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:30.904921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:08:35.901917 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:35.901884    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:08:35.902357 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:35.902058    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:08:49.895936 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:49.895893    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:08:49.896409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:49.895975    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:08:50.985121 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:50.985091    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"]
Sep 14 08:08:50.985616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:50.985298    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" podUID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" containerName="istio-proxy" containerID="cri-o://0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db" gracePeriod=30
Sep 14 08:08:51.052120 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:51.052091    2563 generic.go:356] "Generic (PLEG): container finished" podID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerID="f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592" exitCode=0
Sep 14 08:08:51.052120 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:51.052125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" event={"ID":"07d7fb36-40c1-41e3-94e2-84ac253f0c76","Type":"ContainerDied","Data":"f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"}
Sep 14 08:08:51.052553 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:51.052539    2563 scope.go:117] "RemoveContainer" containerID="f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"
Sep 14 08:08:51.053768 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:51.053720    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:51.053839 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:51.053808    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:51.053878 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:51.053840    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:52.059876 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:52.059844    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" event={"ID":"07d7fb36-40c1-41e3-94e2-84ac253f0c76","Type":"ContainerStarted","Data":"4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45"}
Sep 14 08:08:52.702750 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:52.702695    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:52.703044 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:52.703021    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:08:52.703137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:52.703089    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 14 08:08:53.703273 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:53.703236    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:08:53.703644 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:53.703292    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 14 08:08:54.404782 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.404749    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"]
Sep 14 08:08:54.407423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.407407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:54.409772 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.409753    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:08:54.414765 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.414744    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"]
Sep 14 08:08:54.514339 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.514301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz\" (UID: \"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:54.614972 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.614939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz\" (UID: \"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:54.615611 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.615591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz\" (UID: \"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:54.703174 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.703138    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:08:54.703351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.703201    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 14 08:08:54.704969 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.704944    2563 patch_prober.go:28] interesting pod/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:08:54.705085 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.704997    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" podUID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:08:54.719120 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:54.719101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:55.061644 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:55.061559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"]
Sep 14 08:08:55.069454 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:55.069431    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a43b6fe_4abe_4dd8_b804_5dcb67b48cf4.slice/crio-90388a739c278f8085d57b0dad97ac688d0b52236ad64a64e16d5b2018206574 WatchSource:0}: Error finding container 90388a739c278f8085d57b0dad97ac688d0b52236ad64a64e16d5b2018206574: Status 404 returned error can't find the container with id 90388a739c278f8085d57b0dad97ac688d0b52236ad64a64e16d5b2018206574
Sep 14 08:08:55.082539 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:55.082510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz" event={"ID":"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4","Type":"ContainerStarted","Data":"90388a739c278f8085d57b0dad97ac688d0b52236ad64a64e16d5b2018206574"}
Sep 14 08:08:55.703405 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:55.703325    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:08:55.703853 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:55.703827    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 14 08:08:56.090933 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.090905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz" event={"ID":"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4","Type":"ContainerStarted","Data":"96b4269658df7559f99c517d429b57381d24af51e6ea3317eea8f24220175bf1"}
Sep 14 08:08:56.107363 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.107318    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz" podStartSLOduration=2.107303699 podStartE2EDuration="2.107303699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:56.1054312 +0000 UTC m=+1191.813445341" watchObservedRunningTime="2026-09-14 08:08:56.107303699 +0000 UTC m=+1191.815317838"
Sep 14 08:08:56.282861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.282839    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:08:56.434525 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istiod-ca-cert\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-data\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434620    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-token\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434645    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhr7v\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-kube-api-access-jhr7v\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434671    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-socket\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434743    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-credential-socket\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434812    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-certs\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-podinfo\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434919    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-envoy\") pod \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\" (UID: \"39632f7a-0269-4e67-a239-1a8eb55dd0b8\") "
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434918    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:56.434959 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.434900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:56.435326 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435045    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:56.435326 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:56.435326 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435306    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.435488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435328    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-credential-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.435488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435342    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-workload-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.435488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435356    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istiod-ca-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.435488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.435365    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-data\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.437056 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.437031    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:56.437162 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.437112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:56.437208 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.437188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:08:56.437208 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.437189    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-kube-api-access-jhr7v" (OuterVolumeSpecName: "kube-api-access-jhr7v") pod "39632f7a-0269-4e67-a239-1a8eb55dd0b8" (UID: "39632f7a-0269-4e67-a239-1a8eb55dd0b8"). InnerVolumeSpecName "kube-api-access-jhr7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:56.536449 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.536357    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-podinfo\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.536449 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.536392    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-envoy\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.536449 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.536405    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-istio-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.536449 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.536415    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhr7v\" (UniqueName: \"kubernetes.io/projected/39632f7a-0269-4e67-a239-1a8eb55dd0b8-kube-api-access-jhr7v\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:56.702825 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.702783    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:08:56.702996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:56.702851    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Sep 14 08:08:57.098870 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.098781    2563 generic.go:356] "Generic (PLEG): container finished" podID="0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4" containerID="96b4269658df7559f99c517d429b57381d24af51e6ea3317eea8f24220175bf1" exitCode=0
Sep 14 08:08:57.099216 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.098947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz" event={"ID":"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4","Type":"ContainerDied","Data":"96b4269658df7559f99c517d429b57381d24af51e6ea3317eea8f24220175bf1"}
Sep 14 08:08:57.100252 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.100227    2563 generic.go:356] "Generic (PLEG): container finished" podID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" containerID="0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db" exitCode=0
Sep 14 08:08:57.100369 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.100267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" event={"ID":"39632f7a-0269-4e67-a239-1a8eb55dd0b8","Type":"ContainerDied","Data":"0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db"}
Sep 14 08:08:57.100369 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.100298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr" event={"ID":"39632f7a-0269-4e67-a239-1a8eb55dd0b8","Type":"ContainerDied","Data":"57681d1fade6ae117290a9f72ca513605e9d22b78d5644625ad95ebbc42fd082"}
Sep 14 08:08:57.100369 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.100318    2563 scope.go:117] "RemoveContainer" containerID="0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db"
Sep 14 08:08:57.100475 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.100442    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"
Sep 14 08:08:57.110477 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.110458    2563 scope.go:117] "RemoveContainer" containerID="0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db"
Sep 14 08:08:57.110763 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:08:57.110719    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db\": container with ID starting with 0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db not found: ID does not exist" containerID="0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db"
Sep 14 08:08:57.110833 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.110772    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db"} err="failed to get container status \"0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db\": rpc error: code = NotFound desc = could not find container \"0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db\": container with ID starting with 0625939d60e8301eab4cd4a3a6a7d43f065f5a4737eab1d24c353193c38679db not found: ID does not exist"
Sep 14 08:08:57.130146 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.130114    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"]
Sep 14 08:08:57.139096 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.139073    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-1af772-openshift-default-5b9fb6f4d4-754pr"]
Sep 14 08:08:57.599244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.599205    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"]
Sep 14 08:08:57.599701 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.599686    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" containerName="istio-proxy"
Sep 14 08:08:57.599701 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.599700    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" containerName="istio-proxy"
Sep 14 08:08:57.599910 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.599817    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" containerName="istio-proxy"
Sep 14 08:08:57.603459 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.603431    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.606112 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.606003    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-4d8f28-serving-cert\""
Sep 14 08:08:57.615917 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.615760    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"]
Sep 14 08:08:57.755455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.755419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8582672f-e002-4e1e-bfb2-74e6cdad92c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.755645 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.755486    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.755645 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.755564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr7j2\" (UniqueName: \"kubernetes.io/projected/8582672f-e002-4e1e-bfb2-74e6cdad92c6-kube-api-access-nr7j2\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.755645 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.755630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.770690 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.769928    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:57.770690 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.770327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:57.772392 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.772365    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:08:57.856694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.856609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.856883 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.856693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr7j2\" (UniqueName: \"kubernetes.io/projected/8582672f-e002-4e1e-bfb2-74e6cdad92c6-kube-api-access-nr7j2\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.856883 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.856782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.857230 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.857206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8582672f-e002-4e1e-bfb2-74e6cdad92c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.857851 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.857828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8582672f-e002-4e1e-bfb2-74e6cdad92c6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.859498 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.859460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.860152 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.860126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.865059 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.865034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr7j2\" (UniqueName: \"kubernetes.io/projected/8582672f-e002-4e1e-bfb2-74e6cdad92c6-kube-api-access-nr7j2\") pod \"maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:57.918103 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:57.918077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:58.202186 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.202154    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"]
Sep 14 08:08:58.204670 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:58.204643    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8582672f_e002_4e1e_bfb2_74e6cdad92c6.slice/crio-3f606950e5438e0c1576849cff59b2b741b0187bc6e34b8151a0b4d8af583c61 WatchSource:0}: Error finding container 3f606950e5438e0c1576849cff59b2b741b0187bc6e34b8151a0b4d8af583c61: Status 404 returned error can't find the container with id 3f606950e5438e0c1576849cff59b2b741b0187bc6e34b8151a0b4d8af583c61
Sep 14 08:08:58.268789 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.268762    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:58.326967 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.326927    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"]
Sep 14 08:08:58.327408 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.327391    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4" containerName="revoke-keys"
Sep 14 08:08:58.327408 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.327405    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4" containerName="revoke-keys"
Sep 14 08:08:58.327588 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.327491    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4" containerName="revoke-keys"
Sep 14 08:08:58.329901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.329877    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.332216 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.332195    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-4d8f28-dockercfg-z5ncc\""
Sep 14 08:08:58.332338 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.332202    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-4d8f28\""
Sep 14 08:08:58.342430 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.342403    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"]
Sep 14 08:08:58.362607 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.362547    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4-maas-api-service-ca\") pod \"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4\" (UID: \"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4\") "
Sep 14 08:08:58.362952 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.362931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4" (UID: "0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:08:58.363048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.363035    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:08:58.369661 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.369633    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"]
Sep 14 08:08:58.372247 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.372230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.383109 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.383086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"]
Sep 14 08:08:58.463678 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.463645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppptc\" (UniqueName: \"kubernetes.io/projected/708a9c62-de0d-47b1-b56e-6137be60dd20-kube-api-access-ppptc\") pod \"payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.463869 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.463713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmrcs\" (UniqueName: \"kubernetes.io/projected/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-kube-api-access-mmrcs\") pod \"payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.463869 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.463786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.463869 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.463815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708a9c62-de0d-47b1-b56e-6137be60dd20-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.564783 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.564758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppptc\" (UniqueName: \"kubernetes.io/projected/708a9c62-de0d-47b1-b56e-6137be60dd20-kube-api-access-ppptc\") pod \"payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.564883 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.564826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmrcs\" (UniqueName: \"kubernetes.io/projected/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-kube-api-access-mmrcs\") pod \"payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.564948 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.564895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.565055 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.565036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708a9c62-de0d-47b1-b56e-6137be60dd20-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.565715 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.565686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.565949 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.565928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708a9c62-de0d-47b1-b56e-6137be60dd20-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.572778 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.572751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmrcs\" (UniqueName: \"kubernetes.io/projected/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-kube-api-access-mmrcs\") pod \"payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.573024 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.572999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppptc\" (UniqueName: \"kubernetes.io/projected/708a9c62-de0d-47b1-b56e-6137be60dd20-kube-api-access-ppptc\") pod \"payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.642390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.642310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:58.685225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.685169    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:58.794671 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.794641    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39632f7a-0269-4e67-a239-1a8eb55dd0b8" path="/var/lib/kubelet/pods/39632f7a-0269-4e67-a239-1a8eb55dd0b8/volumes"
Sep 14 08:08:58.921322 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.921289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"]
Sep 14 08:08:58.922182 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:58.922153    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12ec295_3fcb_4d6d_8b5d_b8dae1449866.slice/crio-af7efe43f2b7b6e0e268c2ab14ddfd17a4d3e7ec596d04f625531318d51a36cb WatchSource:0}: Error finding container af7efe43f2b7b6e0e268c2ab14ddfd17a4d3e7ec596d04f625531318d51a36cb: Status 404 returned error can't find the container with id af7efe43f2b7b6e0e268c2ab14ddfd17a4d3e7ec596d04f625531318d51a36cb
Sep 14 08:08:58.976481 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:58.976457    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"]
Sep 14 08:08:58.978548 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:08:58.978523    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708a9c62_de0d_47b1_b56e_6137be60dd20.slice/crio-ab668744b0dc0d99098a5a40f9b8066022cffbaef8b87946c2fe182b9f3b1004 WatchSource:0}: Error finding container ab668744b0dc0d99098a5a40f9b8066022cffbaef8b87946c2fe182b9f3b1004: Status 404 returned error can't find the container with id ab668744b0dc0d99098a5a40f9b8066022cffbaef8b87946c2fe182b9f3b1004
Sep 14 08:08:59.117138 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.117104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g" event={"ID":"8582672f-e002-4e1e-bfb2-74e6cdad92c6","Type":"ContainerStarted","Data":"c59a5911a5e726bcf05a704b919786a47bdae7c195442c20013a228d8e2db0b8"}
Sep 14 08:08:59.117328 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.117191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g" event={"ID":"8582672f-e002-4e1e-bfb2-74e6cdad92c6","Type":"ContainerStarted","Data":"3f606950e5438e0c1576849cff59b2b741b0187bc6e34b8151a0b4d8af583c61"}
Sep 14 08:08:59.117328 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.117219    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:08:59.118488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.118462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz" event={"ID":"0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4","Type":"ContainerDied","Data":"90388a739c278f8085d57b0dad97ac688d0b52236ad64a64e16d5b2018206574"}
Sep 14 08:08:59.118610 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.118508    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90388a739c278f8085d57b0dad97ac688d0b52236ad64a64e16d5b2018206574"
Sep 14 08:08:59.118610 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.118469    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"
Sep 14 08:08:59.120201 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.120177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62" event={"ID":"708a9c62-de0d-47b1-b56e-6137be60dd20","Type":"ContainerStarted","Data":"c982162ddaf3005992e74db8d770effca2a356d1a9bf3b2f397c18a9e4159caf"}
Sep 14 08:08:59.120310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.120221    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62" event={"ID":"708a9c62-de0d-47b1-b56e-6137be60dd20","Type":"ContainerStarted","Data":"ab668744b0dc0d99098a5a40f9b8066022cffbaef8b87946c2fe182b9f3b1004"}
Sep 14 08:08:59.120405 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.120384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:08:59.122144 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.122125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4" event={"ID":"d12ec295-3fcb-4d6d-8b5d-b8dae1449866","Type":"ContainerStarted","Data":"c225e79b5359fbb169cf9397e248fea65f572b547c2a592594a5114a4822e093"}
Sep 14 08:08:59.122248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.122158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4" event={"ID":"d12ec295-3fcb-4d6d-8b5d-b8dae1449866","Type":"ContainerStarted","Data":"af7efe43f2b7b6e0e268c2ab14ddfd17a4d3e7ec596d04f625531318d51a36cb"}
Sep 14 08:08:59.122424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.122407    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:08:59.136654 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.136611    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g" podStartSLOduration=1.828327475 podStartE2EDuration="2.136598266s" totalImagesPullingTime="308.270791ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:59.135400948 +0000 UTC m=+1194.843415087" watchObservedRunningTime="2026-09-14 08:08:59.136598266 +0000 UTC m=+1194.844612447"
Sep 14 08:08:59.155178 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.155126    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4" podStartSLOduration=1.155109213 podStartE2EDuration="1.155109213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:59.152575518 +0000 UTC m=+1194.860589672" watchObservedRunningTime="2026-09-14 08:08:59.155109213 +0000 UTC m=+1194.863123355"
Sep 14 08:08:59.172227 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:08:59.172138    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62" podStartSLOduration=1.172121998 podStartE2EDuration="1.172121998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:59.170609079 +0000 UTC m=+1194.878623221" watchObservedRunningTime="2026-09-14 08:08:59.172121998 +0000 UTC m=+1194.880136142"
Sep 14 08:09:04.772760 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:04.772708    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:09:04.773838 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:04.773817    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:09:05.134277 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:05.134203    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:09:05.401278 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:05.401197    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"]
Sep 14 08:09:05.405652 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:05.405626    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e7b0a8-xwgkz"]
Sep 14 08:09:06.789925 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:06.789897    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4" path="/var/lib/kubelet/pods/0a43b6fe-4abe-4dd8-b804-5dcb67b48cf4/volumes"
Sep 14 08:09:10.130779 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:10.130747    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:09:10.131253 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:10.130925    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:09:23.179575 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.179537    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"]
Sep 14 08:09:23.180302 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.179853    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc" podUID="08dfa45c-65be-431d-a8bf-86d13ecc5ca1" containerName="payload-processing" containerID="cri-o://e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e" gracePeriod=30
Sep 14 08:09:23.183646 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.183618    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"]
Sep 14 08:09:23.184047 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.183998    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl" podUID="1e950b03-d206-4f81-bc2b-b0f0a61a35d1" containerName="payload-pre-processing" containerID="cri-o://d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979" gracePeriod=30
Sep 14 08:09:23.446804 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.446575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:09:23.471776 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.471750    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:09:23.510758 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.510706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk9zl\" (UniqueName: \"kubernetes.io/projected/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-kube-api-access-vk9zl\") pod \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") "
Sep 14 08:09:23.510896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.510882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-plugins-config-volume\") pod \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\" (UID: \"1e950b03-d206-4f81-bc2b-b0f0a61a35d1\") "
Sep 14 08:09:23.511275 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.511246    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1e950b03-d206-4f81-bc2b-b0f0a61a35d1" (UID: "1e950b03-d206-4f81-bc2b-b0f0a61a35d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:23.513233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.513159    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-kube-api-access-vk9zl" (OuterVolumeSpecName: "kube-api-access-vk9zl") pod "1e950b03-d206-4f81-bc2b-b0f0a61a35d1" (UID: "1e950b03-d206-4f81-bc2b-b0f0a61a35d1"). InnerVolumeSpecName "kube-api-access-vk9zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:23.612002 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.611977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvlmp\" (UniqueName: \"kubernetes.io/projected/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-kube-api-access-rvlmp\") pod \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") "
Sep 14 08:09:23.612130 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.612015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-plugins-config-volume\") pod \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\" (UID: \"08dfa45c-65be-431d-a8bf-86d13ecc5ca1\") "
Sep 14 08:09:23.612228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.612217    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.612269 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.612231    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk9zl\" (UniqueName: \"kubernetes.io/projected/1e950b03-d206-4f81-bc2b-b0f0a61a35d1-kube-api-access-vk9zl\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.612420 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.612397    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08dfa45c-65be-431d-a8bf-86d13ecc5ca1" (UID: "08dfa45c-65be-431d-a8bf-86d13ecc5ca1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:23.613989 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.613971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-kube-api-access-rvlmp" (OuterVolumeSpecName: "kube-api-access-rvlmp") pod "08dfa45c-65be-431d-a8bf-86d13ecc5ca1" (UID: "08dfa45c-65be-431d-a8bf-86d13ecc5ca1"). InnerVolumeSpecName "kube-api-access-rvlmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:23.713088 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.713006    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvlmp\" (UniqueName: \"kubernetes.io/projected/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-kube-api-access-rvlmp\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:09:23.713088 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:23.713034    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08dfa45c-65be-431d-a8bf-86d13ecc5ca1-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:09:24.282137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.282099    2563 generic.go:356] "Generic (PLEG): container finished" podID="1e950b03-d206-4f81-bc2b-b0f0a61a35d1" containerID="d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979" exitCode=0
Sep 14 08:09:24.282579 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.282160    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"
Sep 14 08:09:24.282579 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.282188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl" event={"ID":"1e950b03-d206-4f81-bc2b-b0f0a61a35d1","Type":"ContainerDied","Data":"d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979"}
Sep 14 08:09:24.282579 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.282245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl" event={"ID":"1e950b03-d206-4f81-bc2b-b0f0a61a35d1","Type":"ContainerDied","Data":"502d5b88b9d51d93f6ed239e8cb65c0683aa7e6d4c02f4c78573ae15d98a4667"}
Sep 14 08:09:24.282579 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.282274    2563 scope.go:117] "RemoveContainer" containerID="d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979"
Sep 14 08:09:24.283803 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.283781    2563 generic.go:356] "Generic (PLEG): container finished" podID="08dfa45c-65be-431d-a8bf-86d13ecc5ca1" containerID="e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e" exitCode=0
Sep 14 08:09:24.283890 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.283831    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"
Sep 14 08:09:24.283890 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.283868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc" event={"ID":"08dfa45c-65be-431d-a8bf-86d13ecc5ca1","Type":"ContainerDied","Data":"e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e"}
Sep 14 08:09:24.283987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.283922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc" event={"ID":"08dfa45c-65be-431d-a8bf-86d13ecc5ca1","Type":"ContainerDied","Data":"d36ef8394bd5267465a0f3c2da4cda0730bcb79fc9d297f184e7ebf78982f282"}
Sep 14 08:09:24.292073 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.292049    2563 scope.go:117] "RemoveContainer" containerID="d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979"
Sep 14 08:09:24.292378 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:09:24.292358    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979\": container with ID starting with d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979 not found: ID does not exist" containerID="d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979"
Sep 14 08:09:24.292461 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.292385    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979"} err="failed to get container status \"d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979\": rpc error: code = NotFound desc = could not find container \"d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979\": container with ID starting with d02ac64942b8f593f2aca0270fba74961a525b2e6f729ced02ebe9570427f979 not found: ID does not exist"
Sep 14 08:09:24.292461 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.292407    2563 scope.go:117] "RemoveContainer" containerID="e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e"
Sep 14 08:09:24.300685 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.300665    2563 scope.go:117] "RemoveContainer" containerID="e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e"
Sep 14 08:09:24.300961 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:09:24.300944    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e\": container with ID starting with e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e not found: ID does not exist" containerID="e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e"
Sep 14 08:09:24.301016 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.300968    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e"} err="failed to get container status \"e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e\": rpc error: code = NotFound desc = could not find container \"e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e\": container with ID starting with e13c925d7c06a83f11dd6f6270247a7cbd005a15431d616faa54dfc40f85cf1e not found: ID does not exist"
Sep 14 08:09:24.311202 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.311181    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"]
Sep 14 08:09:24.316220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.316190    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f67e60-69d4b9675f-mz8lc"]
Sep 14 08:09:24.327835 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.327804    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"]
Sep 14 08:09:24.334135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.334108    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f67e60-84f47d7966-vxrwl"]
Sep 14 08:09:24.789299 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.789266    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08dfa45c-65be-431d-a8bf-86d13ecc5ca1" path="/var/lib/kubelet/pods/08dfa45c-65be-431d-a8bf-86d13ecc5ca1/volumes"
Sep 14 08:09:24.789618 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:24.789604    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e950b03-d206-4f81-bc2b-b0f0a61a35d1" path="/var/lib/kubelet/pods/1e950b03-d206-4f81-bc2b-b0f0a61a35d1/volumes"
Sep 14 08:09:40.299178 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299142    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"]
Sep 14 08:09:40.299604 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299587    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e950b03-d206-4f81-bc2b-b0f0a61a35d1" containerName="payload-pre-processing"
Sep 14 08:09:40.299670 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299606    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e950b03-d206-4f81-bc2b-b0f0a61a35d1" containerName="payload-pre-processing"
Sep 14 08:09:40.299670 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299634    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08dfa45c-65be-431d-a8bf-86d13ecc5ca1" containerName="payload-processing"
Sep 14 08:09:40.299670 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299640    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="08dfa45c-65be-431d-a8bf-86d13ecc5ca1" containerName="payload-processing"
Sep 14 08:09:40.299877 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299711    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="08dfa45c-65be-431d-a8bf-86d13ecc5ca1" containerName="payload-processing"
Sep 14 08:09:40.299877 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.299736    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e950b03-d206-4f81-bc2b-b0f0a61a35d1" containerName="payload-pre-processing"
Sep 14 08:09:40.302951 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.302934    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.305145 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.305125    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-919cc9\""
Sep 14 08:09:40.305243 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.305182    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-919cc9-dockercfg-tbcfv\""
Sep 14 08:09:40.311471 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.311448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"]
Sep 14 08:09:40.485408 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.485374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db62108-decf-404a-8294-d2bbad8abc63-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.485553 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.485509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r99j2\" (UniqueName: \"kubernetes.io/projected/8db62108-decf-404a-8294-d2bbad8abc63-kube-api-access-r99j2\") pod \"payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.587049 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.586967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db62108-decf-404a-8294-d2bbad8abc63-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.587185 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.587068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r99j2\" (UniqueName: \"kubernetes.io/projected/8db62108-decf-404a-8294-d2bbad8abc63-kube-api-access-r99j2\") pod \"payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.587677 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.587658    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db62108-decf-404a-8294-d2bbad8abc63-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.594463 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.594441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r99j2\" (UniqueName: \"kubernetes.io/projected/8db62108-decf-404a-8294-d2bbad8abc63-kube-api-access-r99j2\") pod \"payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.614774 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.614752    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:40.887032 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:40.887005    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"]
Sep 14 08:09:40.889278 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:09:40.889249    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db62108_decf_404a_8294_d2bbad8abc63.slice/crio-d848b92ea5d7302b4a564317bc84060a0c9e7de85c1d4fd1015aa5759a5080bf WatchSource:0}: Error finding container d848b92ea5d7302b4a564317bc84060a0c9e7de85c1d4fd1015aa5759a5080bf: Status 404 returned error can't find the container with id d848b92ea5d7302b4a564317bc84060a0c9e7de85c1d4fd1015aa5759a5080bf
Sep 14 08:09:41.389534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:41.389492    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv" event={"ID":"8db62108-decf-404a-8294-d2bbad8abc63","Type":"ContainerStarted","Data":"b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5"}
Sep 14 08:09:41.389995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:41.389560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv" event={"ID":"8db62108-decf-404a-8294-d2bbad8abc63","Type":"ContainerStarted","Data":"d848b92ea5d7302b4a564317bc84060a0c9e7de85c1d4fd1015aa5759a5080bf"}
Sep 14 08:09:41.389995 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:41.389720    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:09:41.407430 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:41.407386    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv" podStartSLOduration=1.407370604 podStartE2EDuration="1.407370604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:41.404632603 +0000 UTC m=+1237.112646741" watchObservedRunningTime="2026-09-14 08:09:41.407370604 +0000 UTC m=+1237.115384744"
Sep 14 08:09:52.398908 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:09:52.398877    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:10:04.928534 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:04.928451    2563 scope.go:117] "RemoveContainer" containerID="a06c40478aeeb64288a5e8510c3ed86d5244243a83373f12cd727053fd13abc1"
Sep 14 08:10:04.937688 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:04.937670    2563 scope.go:117] "RemoveContainer" containerID="a51ca77d2c90a5494c3aab6ac4f06e2f4d0aab8506dc2fb693cc6a056f219dd8"
Sep 14 08:10:09.129387 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.129352    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"]
Sep 14 08:10:09.134571 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.134550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.137096 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.137074    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-fc0d6e\""
Sep 14 08:10:09.137392 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.137372    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-fc0d6e-dockercfg-bpfz2\""
Sep 14 08:10:09.144572 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.144548    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"]
Sep 14 08:10:09.163246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.163219    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"]
Sep 14 08:10:09.166927 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.166912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.174588 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.174561    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"]
Sep 14 08:10:09.286710 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.286672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqvw4\" (UniqueName: \"kubernetes.io/projected/20ff6c70-da36-4825-ba6c-3dfaa9505095-kube-api-access-jqvw4\") pod \"payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.286909 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.286760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ff6c70-da36-4825-ba6c-3dfaa9505095-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.286909 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.286790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.286991 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.286934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-kube-api-access-4lt5n\") pod \"payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.388010 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.387927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.388167 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.388063    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-kube-api-access-4lt5n\") pod \"payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.388167 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.388131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqvw4\" (UniqueName: \"kubernetes.io/projected/20ff6c70-da36-4825-ba6c-3dfaa9505095-kube-api-access-jqvw4\") pod \"payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.388274 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.388181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ff6c70-da36-4825-ba6c-3dfaa9505095-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.388928 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.388904    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.388998 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.388939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ff6c70-da36-4825-ba6c-3dfaa9505095-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.395833 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.395805    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-kube-api-access-4lt5n\") pod \"payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.395984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.395963    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqvw4\" (UniqueName: \"kubernetes.io/projected/20ff6c70-da36-4825-ba6c-3dfaa9505095-kube-api-access-jqvw4\") pod \"payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.448867 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.448833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:09.479192 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.479160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:09.722520 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.722495    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"]
Sep 14 08:10:09.724700 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:10:09.724673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20ff6c70_da36_4825_ba6c_3dfaa9505095.slice/crio-a41e2dba8b516bc54de193dab00faec1b76fc21c001121200572f7563f0232be WatchSource:0}: Error finding container a41e2dba8b516bc54de193dab00faec1b76fc21c001121200572f7563f0232be: Status 404 returned error can't find the container with id a41e2dba8b516bc54de193dab00faec1b76fc21c001121200572f7563f0232be
Sep 14 08:10:09.764078 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:09.764049    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"]
Sep 14 08:10:09.765667 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:10:09.765643    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e1a029_bcad_4e62_8f4a_1524f5bb97a0.slice/crio-b4fc9ba051dbad1d697b8a305c73ff9eca1e002efe2f9c8d2942e7a526253111 WatchSource:0}: Error finding container b4fc9ba051dbad1d697b8a305c73ff9eca1e002efe2f9c8d2942e7a526253111: Status 404 returned error can't find the container with id b4fc9ba051dbad1d697b8a305c73ff9eca1e002efe2f9c8d2942e7a526253111
Sep 14 08:10:10.564222 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.564182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh" event={"ID":"20ff6c70-da36-4825-ba6c-3dfaa9505095","Type":"ContainerStarted","Data":"2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94"}
Sep 14 08:10:10.564675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.564287    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:10.564675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.564317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh" event={"ID":"20ff6c70-da36-4825-ba6c-3dfaa9505095","Type":"ContainerStarted","Data":"a41e2dba8b516bc54de193dab00faec1b76fc21c001121200572f7563f0232be"}
Sep 14 08:10:10.565839 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.565812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t" event={"ID":"87e1a029-bcad-4e62-8f4a-1524f5bb97a0","Type":"ContainerStarted","Data":"582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21"}
Sep 14 08:10:10.565942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.565863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t" event={"ID":"87e1a029-bcad-4e62-8f4a-1524f5bb97a0","Type":"ContainerStarted","Data":"b4fc9ba051dbad1d697b8a305c73ff9eca1e002efe2f9c8d2942e7a526253111"}
Sep 14 08:10:10.565942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.565903    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:10.580912 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.580865    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh" podStartSLOduration=1.58085104 podStartE2EDuration="1.58085104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:10:10.579143375 +0000 UTC m=+1266.287157513" watchObservedRunningTime="2026-09-14 08:10:10.58085104 +0000 UTC m=+1266.288865218"
Sep 14 08:10:10.595769 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:10.595699    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t" podStartSLOduration=1.595684313 podStartE2EDuration="1.595684313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:10:10.59323245 +0000 UTC m=+1266.301246589" watchObservedRunningTime="2026-09-14 08:10:10.595684313 +0000 UTC m=+1266.303698452"
Sep 14 08:10:21.575495 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:21.575463    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:10:21.575909 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:21.575574    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:10:41.695283 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.695250    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"]
Sep 14 08:10:41.702849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.702817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.706135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.706108    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2a94f0ce\"/\"default-dockercfg-b5tf5\""
Sep 14 08:10:41.707900 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.707871    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2a94f0ce\"/\"openshift-service-ca.crt\""
Sep 14 08:10:41.708050 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.708031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2a94f0ce\"/\"kube-root-ca.crt\""
Sep 14 08:10:41.708223 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.708044    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2a94f0ce\"/\"e2e-lifecycle-model-2a94f0ce-kserve-self-signed-certs\""
Sep 14 08:10:41.710555 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.710534    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"]
Sep 14 08:10:41.821694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.821666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-model-cache\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.821887 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.821752    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dgdd\" (UniqueName: \"kubernetes.io/projected/b054db39-154f-4305-a187-fef03ddc8dc9-kube-api-access-8dgdd\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.821887 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.821790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-home\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.821887 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.821860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-dshm\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.822056 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.821898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-tmp-dir\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.822056 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.821979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b054db39-154f-4305-a187-fef03ddc8dc9-tls-certs\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b054db39-154f-4305-a187-fef03ddc8dc9-tls-certs\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-model-cache\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dgdd\" (UniqueName: \"kubernetes.io/projected/b054db39-154f-4305-a187-fef03ddc8dc9-kube-api-access-8dgdd\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-home\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-dshm\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923540 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-tmp-dir\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923540 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-model-cache\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-home\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.923718 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.923655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-tmp-dir\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.925429 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.925409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-dshm\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.925867 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.925847    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b054db39-154f-4305-a187-fef03ddc8dc9-tls-certs\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:41.931318 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:41.931293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dgdd\" (UniqueName: \"kubernetes.io/projected/b054db39-154f-4305-a187-fef03ddc8dc9-kube-api-access-8dgdd\") pod \"e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") " pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:42.015669 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:42.015594    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:42.286337 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:42.286314    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"]
Sep 14 08:10:42.288140 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:10:42.288107    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb054db39_154f_4305_a187_fef03ddc8dc9.slice/crio-42f1be4bcd19f12e75321570026dc71a71ce7f7f847b1297a93cfe2620153b79 WatchSource:0}: Error finding container 42f1be4bcd19f12e75321570026dc71a71ce7f7f847b1297a93cfe2620153b79: Status 404 returned error can't find the container with id 42f1be4bcd19f12e75321570026dc71a71ce7f7f847b1297a93cfe2620153b79
Sep 14 08:10:42.765003 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:42.764959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x" event={"ID":"b054db39-154f-4305-a187-fef03ddc8dc9","Type":"ContainerStarted","Data":"6a7b3db73a6e1cc5aec2333f102c1f62264401c379d546c422bc3b3312f15b1a"}
Sep 14 08:10:42.765466 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:42.765029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x" event={"ID":"b054db39-154f-4305-a187-fef03ddc8dc9","Type":"ContainerStarted","Data":"42f1be4bcd19f12e75321570026dc71a71ce7f7f847b1297a93cfe2620153b79"}
Sep 14 08:10:42.783349 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:42.783300    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x" podStartSLOduration=1.783286258 podStartE2EDuration="1.783286258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:10:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:10:42.780876255 +0000 UTC m=+1298.488890387" watchObservedRunningTime="2026-09-14 08:10:42.783286258 +0000 UTC m=+1298.491300397"
Sep 14 08:10:52.016506 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:52.016475    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:52.016911 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:52.016524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:52.029395 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:52.029371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:10:52.833257 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:10:52.833226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:11:04.974720 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:04.974686    2563 scope.go:117] "RemoveContainer" containerID="919657e5bce63799e0aa8db1176ac7450070e3fe4339cca694c0f4e53daec8c2"
Sep 14 08:11:46.053476 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.053442    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"]
Sep 14 08:11:46.057317 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.057299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:46.059592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.059570    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:11:46.064274 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.064245    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"]
Sep 14 08:11:46.138311 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.138270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19611d44-fc1c-4397-8977-6c5a40325756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq\" (UID: \"19611d44-fc1c-4397-8977-6c5a40325756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:46.239337 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.239308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19611d44-fc1c-4397-8977-6c5a40325756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq\" (UID: \"19611d44-fc1c-4397-8977-6c5a40325756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:46.239996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.239972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19611d44-fc1c-4397-8977-6c5a40325756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq\" (UID: \"19611d44-fc1c-4397-8977-6c5a40325756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:46.369509 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.369441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:46.642951 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:46.642924    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"]
Sep 14 08:11:46.645094 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:11:46.645065    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19611d44_fc1c_4397_8977_6c5a40325756.slice/crio-06cf1de2f5c2840ded86117336868530a6d929816fb5e22fd32c3d595e2b02f7 WatchSource:0}: Error finding container 06cf1de2f5c2840ded86117336868530a6d929816fb5e22fd32c3d595e2b02f7: Status 404 returned error can't find the container with id 06cf1de2f5c2840ded86117336868530a6d929816fb5e22fd32c3d595e2b02f7
Sep 14 08:11:47.151139 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:47.151100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq" event={"ID":"19611d44-fc1c-4397-8977-6c5a40325756","Type":"ContainerStarted","Data":"4c537c096dae11423c6571341a34a0dd5c5d8c15ef1b4cabe01774235d2a6fd9"}
Sep 14 08:11:47.151589 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:47.151153    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq" event={"ID":"19611d44-fc1c-4397-8977-6c5a40325756","Type":"ContainerStarted","Data":"06cf1de2f5c2840ded86117336868530a6d929816fb5e22fd32c3d595e2b02f7"}
Sep 14 08:11:47.169321 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:47.169278    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq" podStartSLOduration=1.169261352 podStartE2EDuration="1.169261352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:47.167766997 +0000 UTC m=+1362.875781139" watchObservedRunningTime="2026-09-14 08:11:47.169261352 +0000 UTC m=+1362.877275522"
Sep 14 08:11:48.158354 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:48.158317    2563 generic.go:356] "Generic (PLEG): container finished" podID="19611d44-fc1c-4397-8977-6c5a40325756" containerID="4c537c096dae11423c6571341a34a0dd5c5d8c15ef1b4cabe01774235d2a6fd9" exitCode=0
Sep 14 08:11:48.158758 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:48.158404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq" event={"ID":"19611d44-fc1c-4397-8977-6c5a40325756","Type":"ContainerDied","Data":"4c537c096dae11423c6571341a34a0dd5c5d8c15ef1b4cabe01774235d2a6fd9"}
Sep 14 08:11:49.300139 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:49.300115    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:49.372099 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:49.372071    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19611d44-fc1c-4397-8977-6c5a40325756-maas-api-service-ca\") pod \"19611d44-fc1c-4397-8977-6c5a40325756\" (UID: \"19611d44-fc1c-4397-8977-6c5a40325756\") "
Sep 14 08:11:49.372397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:49.372374    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19611d44-fc1c-4397-8977-6c5a40325756-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19611d44-fc1c-4397-8977-6c5a40325756" (UID: "19611d44-fc1c-4397-8977-6c5a40325756"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:11:49.473937 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:49.473905    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19611d44-fc1c-4397-8977-6c5a40325756-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:11:50.171009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:50.170979    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"
Sep 14 08:11:50.171181 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:50.171023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq" event={"ID":"19611d44-fc1c-4397-8977-6c5a40325756","Type":"ContainerDied","Data":"06cf1de2f5c2840ded86117336868530a6d929816fb5e22fd32c3d595e2b02f7"}
Sep 14 08:11:50.171181 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:11:50.171067    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06cf1de2f5c2840ded86117336868530a6d929816fb5e22fd32c3d595e2b02f7"
Sep 14 08:12:01.742712 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:01.742670    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"]
Sep 14 08:12:01.745135 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:01.745106    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2a94f0ce-x5cvq"]
Sep 14 08:12:02.789976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:02.789940    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19611d44-fc1c-4397-8977-6c5a40325756" path="/var/lib/kubelet/pods/19611d44-fc1c-4397-8977-6c5a40325756/volumes"
Sep 14 08:12:04.996815 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:04.996781    2563 scope.go:117] "RemoveContainer" containerID="b388cf436e843f91362308d9e5eca32eb4c2d0e8133f71c9e3ddaed31b5478c4"
Sep 14 08:12:05.007826 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:05.007803    2563 scope.go:117] "RemoveContainer" containerID="39f6412dde9b5c7850f745b7c441e0e6aac3a6b54accc2673452d2e3abdbde31"
Sep 14 08:12:10.116154 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:10.116119    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"]
Sep 14 08:12:25.136188 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.136143    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x" podUID="b054db39-154f-4305-a187-fef03ddc8dc9" containerName="main" containerID="cri-o://6a7b3db73a6e1cc5aec2333f102c1f62264401c379d546c422bc3b3312f15b1a" gracePeriod=45
Sep 14 08:12:25.378312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.378129    2563 generic.go:356] "Generic (PLEG): container finished" podID="b054db39-154f-4305-a187-fef03ddc8dc9" containerID="6a7b3db73a6e1cc5aec2333f102c1f62264401c379d546c422bc3b3312f15b1a" exitCode=0
Sep 14 08:12:25.378312 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.378180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x" event={"ID":"b054db39-154f-4305-a187-fef03ddc8dc9","Type":"ContainerDied","Data":"6a7b3db73a6e1cc5aec2333f102c1f62264401c379d546c422bc3b3312f15b1a"}
Sep 14 08:12:25.401500 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.401437    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:12:25.543484 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543457    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b054db39-154f-4305-a187-fef03ddc8dc9-tls-certs\") pod \"b054db39-154f-4305-a187-fef03ddc8dc9\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") "
Sep 14 08:12:25.543647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543580    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dgdd\" (UniqueName: \"kubernetes.io/projected/b054db39-154f-4305-a187-fef03ddc8dc9-kube-api-access-8dgdd\") pod \"b054db39-154f-4305-a187-fef03ddc8dc9\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") "
Sep 14 08:12:25.543647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-tmp-dir\") pod \"b054db39-154f-4305-a187-fef03ddc8dc9\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") "
Sep 14 08:12:25.543647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-model-cache\") pod \"b054db39-154f-4305-a187-fef03ddc8dc9\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") "
Sep 14 08:12:25.543830 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543659    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-dshm\") pod \"b054db39-154f-4305-a187-fef03ddc8dc9\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") "
Sep 14 08:12:25.543830 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543769    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-home\") pod \"b054db39-154f-4305-a187-fef03ddc8dc9\" (UID: \"b054db39-154f-4305-a187-fef03ddc8dc9\") "
Sep 14 08:12:25.543972 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543897    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b054db39-154f-4305-a187-fef03ddc8dc9" (UID: "b054db39-154f-4305-a187-fef03ddc8dc9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:25.543972 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.543958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-model-cache" (OuterVolumeSpecName: "model-cache") pod "b054db39-154f-4305-a187-fef03ddc8dc9" (UID: "b054db39-154f-4305-a187-fef03ddc8dc9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:25.544240 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.544144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-home" (OuterVolumeSpecName: "home") pod "b054db39-154f-4305-a187-fef03ddc8dc9" (UID: "b054db39-154f-4305-a187-fef03ddc8dc9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:25.544309 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.544260    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-tmp-dir\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:12:25.544309 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.544280    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-model-cache\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:12:25.544309 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.544292    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-home\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:12:25.545911 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.545889    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-dshm" (OuterVolumeSpecName: "dshm") pod "b054db39-154f-4305-a187-fef03ddc8dc9" (UID: "b054db39-154f-4305-a187-fef03ddc8dc9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:25.546020 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.545999    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b054db39-154f-4305-a187-fef03ddc8dc9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b054db39-154f-4305-a187-fef03ddc8dc9" (UID: "b054db39-154f-4305-a187-fef03ddc8dc9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:25.546093 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.546069    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b054db39-154f-4305-a187-fef03ddc8dc9-kube-api-access-8dgdd" (OuterVolumeSpecName: "kube-api-access-8dgdd") pod "b054db39-154f-4305-a187-fef03ddc8dc9" (UID: "b054db39-154f-4305-a187-fef03ddc8dc9"). InnerVolumeSpecName "kube-api-access-8dgdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:25.645061 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.645028    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b054db39-154f-4305-a187-fef03ddc8dc9-tls-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:12:25.645061 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.645063    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dgdd\" (UniqueName: \"kubernetes.io/projected/b054db39-154f-4305-a187-fef03ddc8dc9-kube-api-access-8dgdd\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:12:25.645256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:25.645078    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b054db39-154f-4305-a187-fef03ddc8dc9-dshm\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:12:26.385809 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:26.385777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x" event={"ID":"b054db39-154f-4305-a187-fef03ddc8dc9","Type":"ContainerDied","Data":"42f1be4bcd19f12e75321570026dc71a71ce7f7f847b1297a93cfe2620153b79"}
Sep 14 08:12:26.385809 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:26.385799    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"
Sep 14 08:12:26.386298 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:26.385823    2563 scope.go:117] "RemoveContainer" containerID="6a7b3db73a6e1cc5aec2333f102c1f62264401c379d546c422bc3b3312f15b1a"
Sep 14 08:12:26.407708 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:26.407683    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"]
Sep 14 08:12:26.410786 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:26.410764    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2a94f0ce/e2e-lifecycle-model-2a94f0ce-kserve-79fbc7cff5-66w6x"]
Sep 14 08:12:26.790607 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:12:26.790578    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b054db39-154f-4305-a187-fef03ddc8dc9" path="/var/lib/kubelet/pods/b054db39-154f-4305-a187-fef03ddc8dc9/volumes"
Sep 14 08:13:13.474511 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.474473    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"]
Sep 14 08:13:13.475021 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.474987    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19611d44-fc1c-4397-8977-6c5a40325756" containerName="revoke-keys"
Sep 14 08:13:13.475021 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.475000    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="19611d44-fc1c-4397-8977-6c5a40325756" containerName="revoke-keys"
Sep 14 08:13:13.475116 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.475030    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b054db39-154f-4305-a187-fef03ddc8dc9" containerName="main"
Sep 14 08:13:13.475116 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.475036    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b054db39-154f-4305-a187-fef03ddc8dc9" containerName="main"
Sep 14 08:13:13.475225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.475118    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b054db39-154f-4305-a187-fef03ddc8dc9" containerName="main"
Sep 14 08:13:13.475225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.475127    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="19611d44-fc1c-4397-8977-6c5a40325756" containerName="revoke-keys"
Sep 14 08:13:13.478991 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.478929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.481326 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.481302    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cd7582\"/\"distinct-2-w4-cd7582-kserve-self-signed-certs\""
Sep 14 08:13:13.488625 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.488211    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"]
Sep 14 08:13:13.497192 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.497160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-home\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.497320 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.497295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433ac3d3-d9df-40f2-b40c-9c902d535f79-tls-certs\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.497396 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.497379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6g87\" (UniqueName: \"kubernetes.io/projected/433ac3d3-d9df-40f2-b40c-9c902d535f79-kube-api-access-q6g87\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.497446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.497435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-model-cache\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.497520 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.497499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-dshm\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.497563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.497529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-tmp-dir\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.598838 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.598805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6g87\" (UniqueName: \"kubernetes.io/projected/433ac3d3-d9df-40f2-b40c-9c902d535f79-kube-api-access-q6g87\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.598987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.598852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-model-cache\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.598987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.598887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-dshm\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.598987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.598904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-tmp-dir\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.598987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.598946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-home\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.599184 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.599024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433ac3d3-d9df-40f2-b40c-9c902d535f79-tls-certs\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.599331 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.599306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-model-cache\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.599331 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.599337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-home\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.599480 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.599365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-tmp-dir\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.601254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.601235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-dshm\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.601460 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.601443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433ac3d3-d9df-40f2-b40c-9c902d535f79-tls-certs\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.605747 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.605701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6g87\" (UniqueName: \"kubernetes.io/projected/433ac3d3-d9df-40f2-b40c-9c902d535f79-kube-api-access-q6g87\") pod \"distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") " pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:13.791751 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:13.791653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:14.064420 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:14.064299    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"]
Sep 14 08:13:14.066973 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:13:14.066942    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod433ac3d3_d9df_40f2_b40c_9c902d535f79.slice/crio-cebe94ed70562b94c3585337bb2be57f588f47d0ee182bab093ef55fe068b503 WatchSource:0}: Error finding container cebe94ed70562b94c3585337bb2be57f588f47d0ee182bab093ef55fe068b503: Status 404 returned error can't find the container with id cebe94ed70562b94c3585337bb2be57f588f47d0ee182bab093ef55fe068b503
Sep 14 08:13:14.685677 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:14.685636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8" event={"ID":"433ac3d3-d9df-40f2-b40c-9c902d535f79","Type":"ContainerStarted","Data":"de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8"}
Sep 14 08:13:14.686149 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:14.686118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8" event={"ID":"433ac3d3-d9df-40f2-b40c-9c902d535f79","Type":"ContainerStarted","Data":"cebe94ed70562b94c3585337bb2be57f588f47d0ee182bab093ef55fe068b503"}
Sep 14 08:13:14.703175 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:14.703126    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8" podStartSLOduration=1.703111651 podStartE2EDuration="1.703111651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:13:14.700919448 +0000 UTC m=+1450.408933588" watchObservedRunningTime="2026-09-14 08:13:14.703111651 +0000 UTC m=+1450.411125789"
Sep 14 08:13:23.792361 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:23.792327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:23.792361 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:23.792361    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:23.804610 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:23.804590    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:13:24.755630 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:13:24.755603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:14:04.812711 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:04.812685    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:14:04.815617 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:04.815084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:14:05.055837 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:05.055803    2563 scope.go:117] "RemoveContainer" containerID="9c523c034b1477e95fb1a8b58cbd3a17d4393eb8b678974ed3c954b49bebba61"
Sep 14 08:14:15.605702 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.605664    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"]
Sep 14 08:14:15.610623 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.610603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:15.613701 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.613677    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:14:15.616618 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.616590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"]
Sep 14 08:14:15.711057 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.711026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0ad783-de12-40ca-bffb-ad730520f409-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw\" (UID: \"cc0ad783-de12-40ca-bffb-ad730520f409\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:15.812364 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.812334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0ad783-de12-40ca-bffb-ad730520f409-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw\" (UID: \"cc0ad783-de12-40ca-bffb-ad730520f409\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:15.812962 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.812945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0ad783-de12-40ca-bffb-ad730520f409-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw\" (UID: \"cc0ad783-de12-40ca-bffb-ad730520f409\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:15.925215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:15.925149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:16.198078 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:16.198053    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"]
Sep 14 08:14:16.200833 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:14:16.200806    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0ad783_de12_40ca_bffb_ad730520f409.slice/crio-14fd53bb99f3a2c898b8d79cab5253ee34d15ca6a142d4976f75bd36e4977a42 WatchSource:0}: Error finding container 14fd53bb99f3a2c898b8d79cab5253ee34d15ca6a142d4976f75bd36e4977a42: Status 404 returned error can't find the container with id 14fd53bb99f3a2c898b8d79cab5253ee34d15ca6a142d4976f75bd36e4977a42
Sep 14 08:14:17.055083 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:17.055044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw" event={"ID":"cc0ad783-de12-40ca-bffb-ad730520f409","Type":"ContainerStarted","Data":"4483f9a801c4610dd25a4118ed2c431841ee6fec16b0e9ac3a69167396eaf895"}
Sep 14 08:14:17.055414 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:17.055155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw" event={"ID":"cc0ad783-de12-40ca-bffb-ad730520f409","Type":"ContainerStarted","Data":"14fd53bb99f3a2c898b8d79cab5253ee34d15ca6a142d4976f75bd36e4977a42"}
Sep 14 08:14:18.063637 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:18.063605    2563 generic.go:356] "Generic (PLEG): container finished" podID="cc0ad783-de12-40ca-bffb-ad730520f409" containerID="4483f9a801c4610dd25a4118ed2c431841ee6fec16b0e9ac3a69167396eaf895" exitCode=0
Sep 14 08:14:18.064095 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:18.063689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw" event={"ID":"cc0ad783-de12-40ca-bffb-ad730520f409","Type":"ContainerDied","Data":"4483f9a801c4610dd25a4118ed2c431841ee6fec16b0e9ac3a69167396eaf895"}
Sep 14 08:14:18.204539 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:18.204461    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:18.344003 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:18.343905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0ad783-de12-40ca-bffb-ad730520f409-maas-api-service-ca\") pod \"cc0ad783-de12-40ca-bffb-ad730520f409\" (UID: \"cc0ad783-de12-40ca-bffb-ad730520f409\") "
Sep 14 08:14:18.344243 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:18.344218    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0ad783-de12-40ca-bffb-ad730520f409-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc0ad783-de12-40ca-bffb-ad730520f409" (UID: "cc0ad783-de12-40ca-bffb-ad730520f409"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:14:18.445365 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:18.445332    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0ad783-de12-40ca-bffb-ad730520f409-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:14:19.070644 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:19.070559    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"
Sep 14 08:14:19.071022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:19.070562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw" event={"ID":"cc0ad783-de12-40ca-bffb-ad730520f409","Type":"ContainerDied","Data":"14fd53bb99f3a2c898b8d79cab5253ee34d15ca6a142d4976f75bd36e4977a42"}
Sep 14 08:14:19.071022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:19.070691    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14fd53bb99f3a2c898b8d79cab5253ee34d15ca6a142d4976f75bd36e4977a42"
Sep 14 08:14:31.450714 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.450618    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"]
Sep 14 08:14:31.451261 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.450898    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg" podUID="770de7b7-117d-40db-b82d-462ac10a9986" containerName="payload-pre-processing" containerID="cri-o://11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6" gracePeriod=30
Sep 14 08:14:31.601589 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.601555    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"]
Sep 14 08:14:31.603898 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.603877    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-831f8e-kq2pw"]
Sep 14 08:14:31.716404 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.716345    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:14:31.774060 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.774026    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/770de7b7-117d-40db-b82d-462ac10a9986-plugins-config-volume\") pod \"770de7b7-117d-40db-b82d-462ac10a9986\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") "
Sep 14 08:14:31.774306 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.774142    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w48m8\" (UniqueName: \"kubernetes.io/projected/770de7b7-117d-40db-b82d-462ac10a9986-kube-api-access-w48m8\") pod \"770de7b7-117d-40db-b82d-462ac10a9986\" (UID: \"770de7b7-117d-40db-b82d-462ac10a9986\") "
Sep 14 08:14:31.774462 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.774435    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/770de7b7-117d-40db-b82d-462ac10a9986-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "770de7b7-117d-40db-b82d-462ac10a9986" (UID: "770de7b7-117d-40db-b82d-462ac10a9986"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:14:31.774572 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.774559    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/770de7b7-117d-40db-b82d-462ac10a9986-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:14:31.776138 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.776118    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/770de7b7-117d-40db-b82d-462ac10a9986-kube-api-access-w48m8" (OuterVolumeSpecName: "kube-api-access-w48m8") pod "770de7b7-117d-40db-b82d-462ac10a9986" (UID: "770de7b7-117d-40db-b82d-462ac10a9986"). InnerVolumeSpecName "kube-api-access-w48m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:31.875238 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:31.875200    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w48m8\" (UniqueName: \"kubernetes.io/projected/770de7b7-117d-40db-b82d-462ac10a9986-kube-api-access-w48m8\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:14:32.146577 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.146495    2563 generic.go:356] "Generic (PLEG): container finished" podID="770de7b7-117d-40db-b82d-462ac10a9986" containerID="11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6" exitCode=0
Sep 14 08:14:32.146577 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.146542    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"
Sep 14 08:14:32.146786 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.146572    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg" event={"ID":"770de7b7-117d-40db-b82d-462ac10a9986","Type":"ContainerDied","Data":"11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6"}
Sep 14 08:14:32.146786 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.146627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg" event={"ID":"770de7b7-117d-40db-b82d-462ac10a9986","Type":"ContainerDied","Data":"963098ccc2f0ca539d2d61c093302306994829ac2b4d6eeddf882a87ce07e6f2"}
Sep 14 08:14:32.146786 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.146648    2563 scope.go:117] "RemoveContainer" containerID="11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6"
Sep 14 08:14:32.156798 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.156779    2563 scope.go:117] "RemoveContainer" containerID="11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6"
Sep 14 08:14:32.157070 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:14:32.157052    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6\": container with ID starting with 11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6 not found: ID does not exist" containerID="11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6"
Sep 14 08:14:32.157144 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.157077    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6"} err="failed to get container status \"11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6\": rpc error: code = NotFound desc = could not find container \"11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6\": container with ID starting with 11f9739a85a2ac4437fddbe76433ffad9332f13caa9f0a11f01c190157e60fd6 not found: ID does not exist"
Sep 14 08:14:32.169569 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.169547    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"]
Sep 14 08:14:32.175980 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.175961    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-831f8e-599c665746-fdcdg"]
Sep 14 08:14:32.789499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.789467    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="770de7b7-117d-40db-b82d-462ac10a9986" path="/var/lib/kubelet/pods/770de7b7-117d-40db-b82d-462ac10a9986/volumes"
Sep 14 08:14:32.789895 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:14:32.789878    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0ad783-de12-40ca-bffb-ad730520f409" path="/var/lib/kubelet/pods/cc0ad783-de12-40ca-bffb-ad730520f409/volumes"
Sep 14 08:15:00.140620 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.140582    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"]
Sep 14 08:15:00.141410 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.141383    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="770de7b7-117d-40db-b82d-462ac10a9986" containerName="payload-pre-processing"
Sep 14 08:15:00.141588 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.141563    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="770de7b7-117d-40db-b82d-462ac10a9986" containerName="payload-pre-processing"
Sep 14 08:15:00.141717 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.141620    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0ad783-de12-40ca-bffb-ad730520f409" containerName="revoke-keys"
Sep 14 08:15:00.141717 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.141629    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0ad783-de12-40ca-bffb-ad730520f409" containerName="revoke-keys"
Sep 14 08:15:00.141856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.141770    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="770de7b7-117d-40db-b82d-462ac10a9986" containerName="payload-pre-processing"
Sep 14 08:15:00.141856 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.141786    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0ad783-de12-40ca-bffb-ad730520f409" containerName="revoke-keys"
Sep 14 08:15:00.145137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.145117    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"]
Sep 14 08:15:00.145256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.145237    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"
Sep 14 08:15:00.147611 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.147586    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:15:00.149183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.149165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"
Sep 14 08:15:00.154294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.154072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"]
Sep 14 08:15:00.157609 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.157528    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"]
Sep 14 08:15:00.159141 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.159120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"
Sep 14 08:15:00.169249 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.169224    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"]
Sep 14 08:15:00.169370 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.169313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"
Sep 14 08:15:00.173707 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.173689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"
Sep 14 08:15:00.181474 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.181449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"]
Sep 14 08:15:00.227693 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.226390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"
Sep 14 08:15:00.448083 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.448053    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"]
Sep 14 08:15:00.450665 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:15:00.450637    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb528e57a_c55b_4f51_a726_b51224b642e6.slice/crio-ac30da13805a647c64f5fe8f2ac06b4c79701da706a1a7e8c7e9889215273377 WatchSource:0}: Error finding container ac30da13805a647c64f5fe8f2ac06b4c79701da706a1a7e8c7e9889215273377: Status 404 returned error can't find the container with id ac30da13805a647c64f5fe8f2ac06b4c79701da706a1a7e8c7e9889215273377
Sep 14 08:15:00.462512 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.462490    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"]
Sep 14 08:15:00.464427 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:15:00.464406    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7942abdd_b3cf_4d8a_98d1_21aa8a850c5f.slice/crio-c03594f10486d16ad521f9b7cdc41cbb730abf46e893e7070d853ae97d54c3eb WatchSource:0}: Error finding container c03594f10486d16ad521f9b7cdc41cbb730abf46e893e7070d853ae97d54c3eb: Status 404 returned error can't find the container with id c03594f10486d16ad521f9b7cdc41cbb730abf46e893e7070d853ae97d54c3eb
Sep 14 08:15:00.521923 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:00.521899    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"]
Sep 14 08:15:00.523944 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:15:00.523923    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod153c7cb9_3dd1_4979_ac8e_63324b2670f1.slice/crio-357c5513e0bd1f1bfd50f65470a056b97bbe48ae549be001ce17ff8d20f0eacf WatchSource:0}: Error finding container 357c5513e0bd1f1bfd50f65470a056b97bbe48ae549be001ce17ff8d20f0eacf: Status 404 returned error can't find the container with id 357c5513e0bd1f1bfd50f65470a056b97bbe48ae549be001ce17ff8d20f0eacf
Sep 14 08:15:01.327503 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.327461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d" event={"ID":"7942abdd-b3cf-4d8a-98d1-21aa8a850c5f","Type":"ContainerStarted","Data":"80028144375cd95cf0a4392223bd102c2178d2bf5a0b4b36378c51315f96c0dd"}
Sep 14 08:15:01.327932 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.327529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d" event={"ID":"7942abdd-b3cf-4d8a-98d1-21aa8a850c5f","Type":"ContainerStarted","Data":"c03594f10486d16ad521f9b7cdc41cbb730abf46e893e7070d853ae97d54c3eb"}
Sep 14 08:15:01.329128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.329098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm" event={"ID":"153c7cb9-3dd1-4979-ac8e-63324b2670f1","Type":"ContainerStarted","Data":"63f103d33364bfa6a0fdc5e6b6801b013ec59ba248dc4f07e9b702943cdc9646"}
Sep 14 08:15:01.329203 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.329147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm" event={"ID":"153c7cb9-3dd1-4979-ac8e-63324b2670f1","Type":"ContainerStarted","Data":"357c5513e0bd1f1bfd50f65470a056b97bbe48ae549be001ce17ff8d20f0eacf"}
Sep 14 08:15:01.333912 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.333885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k" event={"ID":"b528e57a-c55b-4f51-a726-b51224b642e6","Type":"ContainerStarted","Data":"b224c82cfc1f3323ebacdffd70733118fe79331ac7e10698021f011ec80b074c"}
Sep 14 08:15:01.334009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.333932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k" event={"ID":"b528e57a-c55b-4f51-a726-b51224b642e6","Type":"ContainerStarted","Data":"ac30da13805a647c64f5fe8f2ac06b4c79701da706a1a7e8c7e9889215273377"}
Sep 14 08:15:01.343244 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.343201    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d" podStartSLOduration=1.343187172 podStartE2EDuration="1.343187172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:01.340497219 +0000 UTC m=+1557.048511360" watchObservedRunningTime="2026-09-14 08:15:01.343187172 +0000 UTC m=+1557.051201311"
Sep 14 08:15:01.354350 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.354307    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k" podStartSLOduration=1.35429417 podStartE2EDuration="1.35429417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:01.352698718 +0000 UTC m=+1557.060712858" watchObservedRunningTime="2026-09-14 08:15:01.35429417 +0000 UTC m=+1557.062308315"
Sep 14 08:15:01.375339 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:01.375300    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm" podStartSLOduration=1.375287185 podStartE2EDuration="1.375287185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:01.37214025 +0000 UTC m=+1557.080154390" watchObservedRunningTime="2026-09-14 08:15:01.375287185 +0000 UTC m=+1557.083301361"
Sep 14 08:15:02.342434 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:02.342396    2563 generic.go:356] "Generic (PLEG): container finished" podID="b528e57a-c55b-4f51-a726-b51224b642e6" containerID="b224c82cfc1f3323ebacdffd70733118fe79331ac7e10698021f011ec80b074c" exitCode=0
Sep 14 08:15:02.342915 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:02.342487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k" event={"ID":"b528e57a-c55b-4f51-a726-b51224b642e6","Type":"ContainerDied","Data":"b224c82cfc1f3323ebacdffd70733118fe79331ac7e10698021f011ec80b074c"}
Sep 14 08:15:02.344269 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:02.344242    2563 generic.go:356] "Generic (PLEG): container finished" podID="7942abdd-b3cf-4d8a-98d1-21aa8a850c5f" containerID="80028144375cd95cf0a4392223bd102c2178d2bf5a0b4b36378c51315f96c0dd" exitCode=0
Sep 14 08:15:02.344385 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:02.344323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d" event={"ID":"7942abdd-b3cf-4d8a-98d1-21aa8a850c5f","Type":"ContainerDied","Data":"80028144375cd95cf0a4392223bd102c2178d2bf5a0b4b36378c51315f96c0dd"}
Sep 14 08:15:02.346092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:02.346067    2563 generic.go:356] "Generic (PLEG): container finished" podID="153c7cb9-3dd1-4979-ac8e-63324b2670f1" containerID="63f103d33364bfa6a0fdc5e6b6801b013ec59ba248dc4f07e9b702943cdc9646" exitCode=0
Sep 14 08:15:02.346195 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:02.346131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm" event={"ID":"153c7cb9-3dd1-4979-ac8e-63324b2670f1","Type":"ContainerDied","Data":"63f103d33364bfa6a0fdc5e6b6801b013ec59ba248dc4f07e9b702943cdc9646"}
Sep 14 08:15:03.498357 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:03.498330    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"
Sep 14 08:15:03.560260 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:03.560238    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"
Sep 14 08:15:03.563393 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:03.563374    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"
Sep 14 08:15:04.359868 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.359833    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm"
Sep 14 08:15:04.359868 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.359860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-5zszm" event={"ID":"153c7cb9-3dd1-4979-ac8e-63324b2670f1","Type":"ContainerDied","Data":"357c5513e0bd1f1bfd50f65470a056b97bbe48ae549be001ce17ff8d20f0eacf"}
Sep 14 08:15:04.360087 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.359914    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="357c5513e0bd1f1bfd50f65470a056b97bbe48ae549be001ce17ff8d20f0eacf"
Sep 14 08:15:04.361193 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.361173    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"
Sep 14 08:15:04.361291 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.361189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k" event={"ID":"b528e57a-c55b-4f51-a726-b51224b642e6","Type":"ContainerDied","Data":"ac30da13805a647c64f5fe8f2ac06b4c79701da706a1a7e8c7e9889215273377"}
Sep 14 08:15:04.361291 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.361224    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac30da13805a647c64f5fe8f2ac06b4c79701da706a1a7e8c7e9889215273377"
Sep 14 08:15:04.362615 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.362597    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"
Sep 14 08:15:04.362699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.362601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d" event={"ID":"7942abdd-b3cf-4d8a-98d1-21aa8a850c5f","Type":"ContainerDied","Data":"c03594f10486d16ad521f9b7cdc41cbb730abf46e893e7070d853ae97d54c3eb"}
Sep 14 08:15:04.362756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:04.362738    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c03594f10486d16ad521f9b7cdc41cbb730abf46e893e7070d853ae97d54c3eb"
Sep 14 08:15:05.079692 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:05.079659    2563 scope.go:117] "RemoveContainer" containerID="96b4269658df7559f99c517d429b57381d24af51e6ea3317eea8f24220175bf1"
Sep 14 08:15:15.085694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.085659    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"]
Sep 14 08:15:15.086164 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086156    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b528e57a-c55b-4f51-a726-b51224b642e6" containerName="cleanup"
Sep 14 08:15:15.086209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086168    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b528e57a-c55b-4f51-a726-b51224b642e6" containerName="cleanup"
Sep 14 08:15:15.086209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086178    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7942abdd-b3cf-4d8a-98d1-21aa8a850c5f" containerName="cleanup"
Sep 14 08:15:15.086209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086185    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7942abdd-b3cf-4d8a-98d1-21aa8a850c5f" containerName="cleanup"
Sep 14 08:15:15.086209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086199    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="153c7cb9-3dd1-4979-ac8e-63324b2670f1" containerName="cleanup"
Sep 14 08:15:15.086209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086205    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="153c7cb9-3dd1-4979-ac8e-63324b2670f1" containerName="cleanup"
Sep 14 08:15:15.086390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086288    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="153c7cb9-3dd1-4979-ac8e-63324b2670f1" containerName="cleanup"
Sep 14 08:15:15.086390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086297    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7942abdd-b3cf-4d8a-98d1-21aa8a850c5f" containerName="cleanup"
Sep 14 08:15:15.086390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.086305    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b528e57a-c55b-4f51-a726-b51224b642e6" containerName="cleanup"
Sep 14 08:15:15.089644 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.089625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:15.091907 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.091885    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:15:15.097806 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.097779    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"]
Sep 14 08:15:15.212568 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.212532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cabb284-31b5-4d4e-8da8-936cacba1324-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b22414-2hw4s\" (UID: \"9cabb284-31b5-4d4e-8da8-936cacba1324\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:15.313562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.313524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cabb284-31b5-4d4e-8da8-936cacba1324-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b22414-2hw4s\" (UID: \"9cabb284-31b5-4d4e-8da8-936cacba1324\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:15.314261 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.314239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cabb284-31b5-4d4e-8da8-936cacba1324-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b22414-2hw4s\" (UID: \"9cabb284-31b5-4d4e-8da8-936cacba1324\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:15.402896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.402819    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:15.677679 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:15.676160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"]
Sep 14 08:15:15.680203 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:15:15.680170    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cabb284_31b5_4d4e_8da8_936cacba1324.slice/crio-46f4147dcc72169e59418880c662998ce41ca132fcc43195cc1d40dd71f3ef07 WatchSource:0}: Error finding container 46f4147dcc72169e59418880c662998ce41ca132fcc43195cc1d40dd71f3ef07: Status 404 returned error can't find the container with id 46f4147dcc72169e59418880c662998ce41ca132fcc43195cc1d40dd71f3ef07
Sep 14 08:15:16.430259 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:16.430225    2563 generic.go:356] "Generic (PLEG): container finished" podID="9cabb284-31b5-4d4e-8da8-936cacba1324" containerID="ad090dc5354b3e0a62f83ef1049c0ac06150d503be70313b13906190c1d36e74" exitCode=0
Sep 14 08:15:16.430558 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:16.430273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s" event={"ID":"9cabb284-31b5-4d4e-8da8-936cacba1324","Type":"ContainerDied","Data":"ad090dc5354b3e0a62f83ef1049c0ac06150d503be70313b13906190c1d36e74"}
Sep 14 08:15:16.430558 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:16.430323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s" event={"ID":"9cabb284-31b5-4d4e-8da8-936cacba1324","Type":"ContainerStarted","Data":"46f4147dcc72169e59418880c662998ce41ca132fcc43195cc1d40dd71f3ef07"}
Sep 14 08:15:17.583089 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:17.583062    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:17.636005 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:17.635970    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cabb284-31b5-4d4e-8da8-936cacba1324-maas-api-service-ca\") pod \"9cabb284-31b5-4d4e-8da8-936cacba1324\" (UID: \"9cabb284-31b5-4d4e-8da8-936cacba1324\") "
Sep 14 08:15:17.636305 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:17.636282    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cabb284-31b5-4d4e-8da8-936cacba1324-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9cabb284-31b5-4d4e-8da8-936cacba1324" (UID: "9cabb284-31b5-4d4e-8da8-936cacba1324"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:15:17.737597 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:17.737527    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cabb284-31b5-4d4e-8da8-936cacba1324-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:15:18.445430 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:18.445399    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"
Sep 14 08:15:18.445598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:18.445440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s" event={"ID":"9cabb284-31b5-4d4e-8da8-936cacba1324","Type":"ContainerDied","Data":"46f4147dcc72169e59418880c662998ce41ca132fcc43195cc1d40dd71f3ef07"}
Sep 14 08:15:18.445598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:18.445496    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="46f4147dcc72169e59418880c662998ce41ca132fcc43195cc1d40dd71f3ef07"
Sep 14 08:15:31.180504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:31.180466    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"]
Sep 14 08:15:31.182733 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:31.182698    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b22414-2hw4s"]
Sep 14 08:15:32.790103 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:32.790072    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cabb284-31b5-4d4e-8da8-936cacba1324" path="/var/lib/kubelet/pods/9cabb284-31b5-4d4e-8da8-936cacba1324/volumes"
Sep 14 08:15:53.861241 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:53.861210    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"]
Sep 14 08:15:53.863488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:53.863465    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e721c-29822895-pxd7d"]
Sep 14 08:15:54.792754 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:15:54.791282    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7942abdd-b3cf-4d8a-98d1-21aa8a850c5f" path="/var/lib/kubelet/pods/7942abdd-b3cf-4d8a-98d1-21aa8a850c5f/volumes"
Sep 14 08:16:00.305942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:00.305858    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"]
Sep 14 08:16:00.319115 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:00.319083    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"]
Sep 14 08:16:11.679417 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.679377    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"]
Sep 14 08:16:11.679923 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.679595    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh" podUID="2eaa106f-9457-45f8-b699-a2c95aadb1ae" containerName="payload-pre-processing" containerID="cri-o://2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378" gracePeriod=30
Sep 14 08:16:11.766504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.766468    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"]
Sep 14 08:16:11.766770 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.766711    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" podUID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea" containerName="maas-api" containerID="cri-o://9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2" gracePeriod=30
Sep 14 08:16:11.824445 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.824411    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"]
Sep 14 08:16:11.824713 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.824660    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72" podUID="669ebfb4-dc18-4ef5-99c4-0d626661bff7" containerName="payload-processing" containerID="cri-o://fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1" gracePeriod=30
Sep 14 08:16:11.834171 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:11.834087    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"]
Sep 14 08:16:12.029330 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.029307    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:16:12.073442 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.073420    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngqcx\" (UniqueName: \"kubernetes.io/projected/2eaa106f-9457-45f8-b699-a2c95aadb1ae-kube-api-access-ngqcx\") pod \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") "
Sep 14 08:16:12.073598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.073516    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eaa106f-9457-45f8-b699-a2c95aadb1ae-plugins-config-volume\") pod \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\" (UID: \"2eaa106f-9457-45f8-b699-a2c95aadb1ae\") "
Sep 14 08:16:12.074006 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.073974    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eaa106f-9457-45f8-b699-a2c95aadb1ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2eaa106f-9457-45f8-b699-a2c95aadb1ae" (UID: "2eaa106f-9457-45f8-b699-a2c95aadb1ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:12.075971 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.075941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2eaa106f-9457-45f8-b699-a2c95aadb1ae-kube-api-access-ngqcx" (OuterVolumeSpecName: "kube-api-access-ngqcx") pod "2eaa106f-9457-45f8-b699-a2c95aadb1ae" (UID: "2eaa106f-9457-45f8-b699-a2c95aadb1ae"). InnerVolumeSpecName "kube-api-access-ngqcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:12.077401 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.077377    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:16:12.121665 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.121643    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:16:12.174822 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.174779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-openshift-service-ca\") pod \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") "
Sep 14 08:16:12.175018 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.174834    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-metrics-certs\") pod \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") "
Sep 14 08:16:12.175018 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.174945    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvhcs\" (UniqueName: \"kubernetes.io/projected/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-kube-api-access-fvhcs\") pod \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") "
Sep 14 08:16:12.175018 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.174980    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls\") pod \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\" (UID: \"3a3dcf97-eb10-4dc1-92a8-497fd83363ea\") "
Sep 14 08:16:12.175161 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.175048    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/669ebfb4-dc18-4ef5-99c4-0d626661bff7-plugins-config-volume\") pod \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") "
Sep 14 08:16:12.175161 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.175088    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qlrr\" (UniqueName: \"kubernetes.io/projected/669ebfb4-dc18-4ef5-99c4-0d626661bff7-kube-api-access-6qlrr\") pod \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\" (UID: \"669ebfb4-dc18-4ef5-99c4-0d626661bff7\") "
Sep 14 08:16:12.175258 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.175222    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3a3dcf97-eb10-4dc1-92a8-497fd83363ea" (UID: "3a3dcf97-eb10-4dc1-92a8-497fd83363ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:12.175702 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.175521    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngqcx\" (UniqueName: \"kubernetes.io/projected/2eaa106f-9457-45f8-b699-a2c95aadb1ae-kube-api-access-ngqcx\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.175702 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.175543    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2eaa106f-9457-45f8-b699-a2c95aadb1ae-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.175702 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.175556    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.176459 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.176245    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/669ebfb4-dc18-4ef5-99c4-0d626661bff7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "669ebfb4-dc18-4ef5-99c4-0d626661bff7" (UID: "669ebfb4-dc18-4ef5-99c4-0d626661bff7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:12.177636 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.177607    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-kube-api-access-fvhcs" (OuterVolumeSpecName: "kube-api-access-fvhcs") pod "3a3dcf97-eb10-4dc1-92a8-497fd83363ea" (UID: "3a3dcf97-eb10-4dc1-92a8-497fd83363ea"). InnerVolumeSpecName "kube-api-access-fvhcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:12.178128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.178101    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/669ebfb4-dc18-4ef5-99c4-0d626661bff7-kube-api-access-6qlrr" (OuterVolumeSpecName: "kube-api-access-6qlrr") pod "669ebfb4-dc18-4ef5-99c4-0d626661bff7" (UID: "669ebfb4-dc18-4ef5-99c4-0d626661bff7"). InnerVolumeSpecName "kube-api-access-6qlrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:12.178265 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.178241    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3a3dcf97-eb10-4dc1-92a8-497fd83363ea" (UID: "3a3dcf97-eb10-4dc1-92a8-497fd83363ea"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:12.178402 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.178375    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3a3dcf97-eb10-4dc1-92a8-497fd83363ea" (UID: "3a3dcf97-eb10-4dc1-92a8-497fd83363ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:12.276283 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.276206    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.276283 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.276233    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvhcs\" (UniqueName: \"kubernetes.io/projected/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-kube-api-access-fvhcs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.276283 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.276245    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a3dcf97-eb10-4dc1-92a8-497fd83363ea-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.276283 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.276254    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/669ebfb4-dc18-4ef5-99c4-0d626661bff7-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.276283 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.276262    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qlrr\" (UniqueName: \"kubernetes.io/projected/669ebfb4-dc18-4ef5-99c4-0d626661bff7-kube-api-access-6qlrr\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:12.768430 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.768398    2563 generic.go:356] "Generic (PLEG): container finished" podID="2eaa106f-9457-45f8-b699-a2c95aadb1ae" containerID="2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378" exitCode=0
Sep 14 08:16:12.768871 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.768447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh" event={"ID":"2eaa106f-9457-45f8-b699-a2c95aadb1ae","Type":"ContainerDied","Data":"2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378"}
Sep 14 08:16:12.768871 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.768456    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"
Sep 14 08:16:12.768871 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.768537    2563 scope.go:117] "RemoveContainer" containerID="2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378"
Sep 14 08:16:12.768871 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.768521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh" event={"ID":"2eaa106f-9457-45f8-b699-a2c95aadb1ae","Type":"ContainerDied","Data":"4dc671f7b49420306c96eca4e1380edddcea735ba48bfb63e95978e81105039a"}
Sep 14 08:16:12.770351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.770128    2563 generic.go:356] "Generic (PLEG): container finished" podID="669ebfb4-dc18-4ef5-99c4-0d626661bff7" containerID="fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1" exitCode=0
Sep 14 08:16:12.770351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.770220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72" event={"ID":"669ebfb4-dc18-4ef5-99c4-0d626661bff7","Type":"ContainerDied","Data":"fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1"}
Sep 14 08:16:12.770351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.770257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72" event={"ID":"669ebfb4-dc18-4ef5-99c4-0d626661bff7","Type":"ContainerDied","Data":"55f46889335aa257f5b486238f6edd7e386598fc0524a5f797991357bc5cb8e3"}
Sep 14 08:16:12.770351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.770334    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"
Sep 14 08:16:12.773254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.772412    2563 generic.go:356] "Generic (PLEG): container finished" podID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea" containerID="9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2" exitCode=0
Sep 14 08:16:12.773254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.772496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:16:12.773254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.772526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" event={"ID":"3a3dcf97-eb10-4dc1-92a8-497fd83363ea","Type":"ContainerDied","Data":"9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2"}
Sep 14 08:16:12.773254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.772565    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" event={"ID":"3a3dcf97-eb10-4dc1-92a8-497fd83363ea","Type":"ContainerDied","Data":"dfc6976b6fd945d7d2881455342f5963eb18f061d9891e7ec8869bf64640a930"}
Sep 14 08:16:12.773254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.772580    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v" podUID="06ea5649-56da-4ebf-9515-da506c65cd74" containerName="payload-pre-processing" containerID="cri-o://342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73" gracePeriod=30
Sep 14 08:16:12.781082 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.781065    2563 scope.go:117] "RemoveContainer" containerID="2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378"
Sep 14 08:16:12.781538 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:12.781387    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378\": container with ID starting with 2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378 not found: ID does not exist" containerID="2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378"
Sep 14 08:16:12.781538 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.781424    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378"} err="failed to get container status \"2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378\": rpc error: code = NotFound desc = could not find container \"2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378\": container with ID starting with 2ef23f6f5d64b4b1e342438ad8745c5f8b1e1a2f9103144f3dc0b0d64063d378 not found: ID does not exist"
Sep 14 08:16:12.781538 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.781447    2563 scope.go:117] "RemoveContainer" containerID="fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1"
Sep 14 08:16:12.844846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.844822    2563 scope.go:117] "RemoveContainer" containerID="fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1"
Sep 14 08:16:12.845212 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:12.845183    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1\": container with ID starting with fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1 not found: ID does not exist" containerID="fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1"
Sep 14 08:16:12.845290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.845224    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1"} err="failed to get container status \"fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1\": rpc error: code = NotFound desc = could not find container \"fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1\": container with ID starting with fd37e2527d8ef46e8069b9630f6488fa9b5fb3f25e7e423425646ec0fc83a0f1 not found: ID does not exist"
Sep 14 08:16:12.845290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.845250    2563 scope.go:117] "RemoveContainer" containerID="9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2"
Sep 14 08:16:12.851294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.851272    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"]
Sep 14 08:16:12.855443 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.855420    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cd7582-7946cdcf46-2kpkh"]
Sep 14 08:16:12.870446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.870423    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"]
Sep 14 08:16:12.873868 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.873848    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-0b8346-799cb5956c-pxs72"]
Sep 14 08:16:12.884974 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.884952    2563 scope.go:117] "RemoveContainer" containerID="9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2"
Sep 14 08:16:12.885257 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:12.885234    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2\": container with ID starting with 9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2 not found: ID does not exist" containerID="9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2"
Sep 14 08:16:12.885325 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:12.885269    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2"} err="failed to get container status \"9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2\": rpc error: code = NotFound desc = could not find container \"9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2\": container with ID starting with 9ca7deab48ca1cd468d392df75e5d2ef37b6d04c88f208e26901194d6d24d8f2 not found: ID does not exist"
Sep 14 08:16:13.027120 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.027096    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:16:13.083546 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.083515    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2w77t\" (UniqueName: \"kubernetes.io/projected/06ea5649-56da-4ebf-9515-da506c65cd74-kube-api-access-2w77t\") pod \"06ea5649-56da-4ebf-9515-da506c65cd74\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") "
Sep 14 08:16:13.083706 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.083553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ea5649-56da-4ebf-9515-da506c65cd74-plugins-config-volume\") pod \"06ea5649-56da-4ebf-9515-da506c65cd74\" (UID: \"06ea5649-56da-4ebf-9515-da506c65cd74\") "
Sep 14 08:16:13.084009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.083986    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06ea5649-56da-4ebf-9515-da506c65cd74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06ea5649-56da-4ebf-9515-da506c65cd74" (UID: "06ea5649-56da-4ebf-9515-da506c65cd74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:13.085811 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.085786    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ea5649-56da-4ebf-9515-da506c65cd74-kube-api-access-2w77t" (OuterVolumeSpecName: "kube-api-access-2w77t") pod "06ea5649-56da-4ebf-9515-da506c65cd74" (UID: "06ea5649-56da-4ebf-9515-da506c65cd74"). InnerVolumeSpecName "kube-api-access-2w77t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:13.185230 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.185196    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w77t\" (UniqueName: \"kubernetes.io/projected/06ea5649-56da-4ebf-9515-da506c65cd74-kube-api-access-2w77t\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:13.185230 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.185226    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06ea5649-56da-4ebf-9515-da506c65cd74-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:13.782122 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.782089    2563 generic.go:356] "Generic (PLEG): container finished" podID="06ea5649-56da-4ebf-9515-da506c65cd74" containerID="342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73" exitCode=0
Sep 14 08:16:13.782543 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.782170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v" event={"ID":"06ea5649-56da-4ebf-9515-da506c65cd74","Type":"ContainerDied","Data":"342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73"}
Sep 14 08:16:13.782543 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.782213    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v" event={"ID":"06ea5649-56da-4ebf-9515-da506c65cd74","Type":"ContainerDied","Data":"223dc8bdbab1e798bd538f2ac940942104ea44f724707f7f44558da2a43399a5"}
Sep 14 08:16:13.782543 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.782234    2563 scope.go:117] "RemoveContainer" containerID="342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73"
Sep 14 08:16:13.782543 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.782248    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"
Sep 14 08:16:13.791847 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.791831    2563 scope.go:117] "RemoveContainer" containerID="342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73"
Sep 14 08:16:13.792105 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:13.792085    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73\": container with ID starting with 342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73 not found: ID does not exist" containerID="342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73"
Sep 14 08:16:13.792151 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.792113    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73"} err="failed to get container status \"342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73\": rpc error: code = NotFound desc = could not find container \"342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73\": container with ID starting with 342b627a54d338e44a839925996aa907139f10cd9fcb4ea65ea422f90c651c73 not found: ID does not exist"
Sep 14 08:16:13.807080 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.807055    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"]
Sep 14 08:16:13.810834 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:13.810812    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0b8346-7f48c8c54-kh88v"]
Sep 14 08:16:14.790101 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:14.790071    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06ea5649-56da-4ebf-9515-da506c65cd74" path="/var/lib/kubelet/pods/06ea5649-56da-4ebf-9515-da506c65cd74/volumes"
Sep 14 08:16:14.790486 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:14.790442    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eaa106f-9457-45f8-b699-a2c95aadb1ae" path="/var/lib/kubelet/pods/2eaa106f-9457-45f8-b699-a2c95aadb1ae/volumes"
Sep 14 08:16:14.790785 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:14.790771    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="669ebfb4-dc18-4ef5-99c4-0d626661bff7" path="/var/lib/kubelet/pods/669ebfb4-dc18-4ef5-99c4-0d626661bff7/volumes"
Sep 14 08:16:15.338565 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.338494    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt" podUID="8540d7c2-55c1-4e92-b260-b00a2ae89591" containerName="main" containerID="cri-o://6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b" gracePeriod=45
Sep 14 08:16:15.592781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.592711    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:16:15.609811 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.609786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-home\") pod \"8540d7c2-55c1-4e92-b260-b00a2ae89591\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") "
Sep 14 08:16:15.609949 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.609820    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-model-cache\") pod \"8540d7c2-55c1-4e92-b260-b00a2ae89591\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") "
Sep 14 08:16:15.609949 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.609917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkgqt\" (UniqueName: \"kubernetes.io/projected/8540d7c2-55c1-4e92-b260-b00a2ae89591-kube-api-access-fkgqt\") pod \"8540d7c2-55c1-4e92-b260-b00a2ae89591\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") "
Sep 14 08:16:15.609949 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.609948    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-tmp-dir\") pod \"8540d7c2-55c1-4e92-b260-b00a2ae89591\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") "
Sep 14 08:16:15.610116 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.609981    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-dshm\") pod \"8540d7c2-55c1-4e92-b260-b00a2ae89591\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") "
Sep 14 08:16:15.610116 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610080    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-home" (OuterVolumeSpecName: "home") pod "8540d7c2-55c1-4e92-b260-b00a2ae89591" (UID: "8540d7c2-55c1-4e92-b260-b00a2ae89591"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.610196 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610128    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-model-cache" (OuterVolumeSpecName: "model-cache") pod "8540d7c2-55c1-4e92-b260-b00a2ae89591" (UID: "8540d7c2-55c1-4e92-b260-b00a2ae89591"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.610248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610227    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8540d7c2-55c1-4e92-b260-b00a2ae89591" (UID: "8540d7c2-55c1-4e92-b260-b00a2ae89591"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.610357 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8540d7c2-55c1-4e92-b260-b00a2ae89591-tls-certs\") pod \"8540d7c2-55c1-4e92-b260-b00a2ae89591\" (UID: \"8540d7c2-55c1-4e92-b260-b00a2ae89591\") "
Sep 14 08:16:15.610806 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610784    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-home\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.610907 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610810    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-model-cache\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.610907 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.610824    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-tmp-dir\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.612453 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.612419    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8540d7c2-55c1-4e92-b260-b00a2ae89591-kube-api-access-fkgqt" (OuterVolumeSpecName: "kube-api-access-fkgqt") pod "8540d7c2-55c1-4e92-b260-b00a2ae89591" (UID: "8540d7c2-55c1-4e92-b260-b00a2ae89591"). InnerVolumeSpecName "kube-api-access-fkgqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:15.612548 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.612466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-dshm" (OuterVolumeSpecName: "dshm") pod "8540d7c2-55c1-4e92-b260-b00a2ae89591" (UID: "8540d7c2-55c1-4e92-b260-b00a2ae89591"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:15.612781 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.612754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8540d7c2-55c1-4e92-b260-b00a2ae89591-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8540d7c2-55c1-4e92-b260-b00a2ae89591" (UID: "8540d7c2-55c1-4e92-b260-b00a2ae89591"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:15.710866 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.710825    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8" podUID="433ac3d3-d9df-40f2-b40c-9c902d535f79" containerName="main" containerID="cri-o://de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8" gracePeriod=45
Sep 14 08:16:15.711389 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.711366    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8540d7c2-55c1-4e92-b260-b00a2ae89591-dshm\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.711446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.711392    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8540d7c2-55c1-4e92-b260-b00a2ae89591-tls-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.711446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.711403    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkgqt\" (UniqueName: \"kubernetes.io/projected/8540d7c2-55c1-4e92-b260-b00a2ae89591-kube-api-access-fkgqt\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:15.796609 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.796573    2563 generic.go:356] "Generic (PLEG): container finished" podID="8540d7c2-55c1-4e92-b260-b00a2ae89591" containerID="6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b" exitCode=0
Sep 14 08:16:15.796985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.796618    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"
Sep 14 08:16:15.796985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.796656    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt" event={"ID":"8540d7c2-55c1-4e92-b260-b00a2ae89591","Type":"ContainerDied","Data":"6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b"}
Sep 14 08:16:15.796985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.796700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt" event={"ID":"8540d7c2-55c1-4e92-b260-b00a2ae89591","Type":"ContainerDied","Data":"c98c0c4be84c1351e240bd520de7b375f742ed965335ba3dcd83e1568a98ef22"}
Sep 14 08:16:15.796985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.796716    2563 scope.go:117] "RemoveContainer" containerID="6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b"
Sep 14 08:16:15.823008 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.822812    2563 scope.go:117] "RemoveContainer" containerID="6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b"
Sep 14 08:16:15.823170 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:15.823144    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b\": container with ID starting with 6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b not found: ID does not exist" containerID="6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b"
Sep 14 08:16:15.823295 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.823174    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b"} err="failed to get container status \"6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b\": rpc error: code = NotFound desc = could not find container \"6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b\": container with ID starting with 6d41a6c609354bf067abeceda35f52a4e09be5e2462dab69be9a65eb8febe00b not found: ID does not exist"
Sep 14 08:16:15.825279 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.825253    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"]
Sep 14 08:16:15.828424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.828403    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/premium-w4-cd7582-kserve-86f769885f-mvlpt"]
Sep 14 08:16:15.945583 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:15.945555    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:16:16.013675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-model-cache\") pod \"433ac3d3-d9df-40f2-b40c-9c902d535f79\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") "
Sep 14 08:16:16.013862 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013687    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-tmp-dir\") pod \"433ac3d3-d9df-40f2-b40c-9c902d535f79\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") "
Sep 14 08:16:16.013862 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013721    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-home\") pod \"433ac3d3-d9df-40f2-b40c-9c902d535f79\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") "
Sep 14 08:16:16.013862 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013808    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6g87\" (UniqueName: \"kubernetes.io/projected/433ac3d3-d9df-40f2-b40c-9c902d535f79-kube-api-access-q6g87\") pod \"433ac3d3-d9df-40f2-b40c-9c902d535f79\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") "
Sep 14 08:16:16.014045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013879    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-dshm\") pod \"433ac3d3-d9df-40f2-b40c-9c902d535f79\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") "
Sep 14 08:16:16.014045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433ac3d3-d9df-40f2-b40c-9c902d535f79-tls-certs\") pod \"433ac3d3-d9df-40f2-b40c-9c902d535f79\" (UID: \"433ac3d3-d9df-40f2-b40c-9c902d535f79\") "
Sep 14 08:16:16.014045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-model-cache" (OuterVolumeSpecName: "model-cache") pod "433ac3d3-d9df-40f2-b40c-9c902d535f79" (UID: "433ac3d3-d9df-40f2-b40c-9c902d535f79"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:16.014045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.013959    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "433ac3d3-d9df-40f2-b40c-9c902d535f79" (UID: "433ac3d3-d9df-40f2-b40c-9c902d535f79"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:16.014045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.014028    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-home" (OuterVolumeSpecName: "home") pod "433ac3d3-d9df-40f2-b40c-9c902d535f79" (UID: "433ac3d3-d9df-40f2-b40c-9c902d535f79"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:16.014320 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.014301    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-model-cache\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.014379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.014324    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-tmp-dir\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.014379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.014339    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-home\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.015917 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.015894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-dshm" (OuterVolumeSpecName: "dshm") pod "433ac3d3-d9df-40f2-b40c-9c902d535f79" (UID: "433ac3d3-d9df-40f2-b40c-9c902d535f79"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:16.016033 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.015898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/433ac3d3-d9df-40f2-b40c-9c902d535f79-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "433ac3d3-d9df-40f2-b40c-9c902d535f79" (UID: "433ac3d3-d9df-40f2-b40c-9c902d535f79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:16.016033 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.015989    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/433ac3d3-d9df-40f2-b40c-9c902d535f79-kube-api-access-q6g87" (OuterVolumeSpecName: "kube-api-access-q6g87") pod "433ac3d3-d9df-40f2-b40c-9c902d535f79" (UID: "433ac3d3-d9df-40f2-b40c-9c902d535f79"). InnerVolumeSpecName "kube-api-access-q6g87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:16.115016 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.114987    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6g87\" (UniqueName: \"kubernetes.io/projected/433ac3d3-d9df-40f2-b40c-9c902d535f79-kube-api-access-q6g87\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.115016 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.115011    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/433ac3d3-d9df-40f2-b40c-9c902d535f79-dshm\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.115016 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.115021    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/433ac3d3-d9df-40f2-b40c-9c902d535f79-tls-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:16.791256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.791221    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8540d7c2-55c1-4e92-b260-b00a2ae89591" path="/var/lib/kubelet/pods/8540d7c2-55c1-4e92-b260-b00a2ae89591/volumes"
Sep 14 08:16:16.803905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.803881    2563 generic.go:356] "Generic (PLEG): container finished" podID="433ac3d3-d9df-40f2-b40c-9c902d535f79" containerID="de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8" exitCode=0
Sep 14 08:16:16.804258 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.803920    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"
Sep 14 08:16:16.804258 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.803958    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8" event={"ID":"433ac3d3-d9df-40f2-b40c-9c902d535f79","Type":"ContainerDied","Data":"de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8"}
Sep 14 08:16:16.804258 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.803995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8" event={"ID":"433ac3d3-d9df-40f2-b40c-9c902d535f79","Type":"ContainerDied","Data":"cebe94ed70562b94c3585337bb2be57f588f47d0ee182bab093ef55fe068b503"}
Sep 14 08:16:16.804258 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.804010    2563 scope.go:117] "RemoveContainer" containerID="de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8"
Sep 14 08:16:16.812415 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.812399    2563 scope.go:117] "RemoveContainer" containerID="de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8"
Sep 14 08:16:16.812670 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:16.812646    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8\": container with ID starting with de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8 not found: ID does not exist" containerID="de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8"
Sep 14 08:16:16.812740 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.812678    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8"} err="failed to get container status \"de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8\": rpc error: code = NotFound desc = could not find container \"de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8\": container with ID starting with de017fb2131f73dd980995c6eab3d726bbca0d2bc967506c0f3512d0e72620d8 not found: ID does not exist"
Sep 14 08:16:16.818963 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.818943    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"]
Sep 14 08:16:16.822308 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:16.822282    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cd7582/distinct-2-w4-cd7582-kserve-d58876fd5-w7nc8"]
Sep 14 08:16:18.789539 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:18.789499    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="433ac3d3-d9df-40f2-b40c-9c902d535f79" path="/var/lib/kubelet/pods/433ac3d3-d9df-40f2-b40c-9c902d535f79/volumes"
Sep 14 08:16:19.784209 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:19.784178    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"]
Sep 14 08:16:19.784402 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:19.784380    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy" containerID="cri-o://4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45" gracePeriod=30
Sep 14 08:16:25.047950 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.047888    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:16:25.100390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100367    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-socket\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100516 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-certs\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100516 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100431    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbh45\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-kube-api-access-lbh45\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100516 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100451    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-token\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100529    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-data\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-credential-socket\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100573    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-podinfo\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100591    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istiod-ca-cert\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100612    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-envoy\") pod \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\" (UID: \"07d7fb36-40c1-41e3-94e2-84ac253f0c76\") "
Sep 14 08:16:25.100691 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:25.101079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.100874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-data" (OuterVolumeSpecName: "istio-data") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:25.101079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.101012    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:25.101201 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.101127    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:25.101201 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.101192    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.101309 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.101212    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-workload-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.101309 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.101227    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-data\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.101309 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.101239    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istiod-ca-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.102833 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.102807    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-kube-api-access-lbh45" (OuterVolumeSpecName: "kube-api-access-lbh45") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "kube-api-access-lbh45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:25.102833 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.102820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-token" (OuterVolumeSpecName: "istio-token") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:25.103067 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.103048    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:25.103124 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.103068    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07d7fb36-40c1-41e3-94e2-84ac253f0c76" (UID: "07d7fb36-40c1-41e3-94e2-84ac253f0c76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:16:25.202121 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.202094    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-envoy\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.202121 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.202117    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbh45\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-kube-api-access-lbh45\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.202121 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.202127    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.202328 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.202136    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07d7fb36-40c1-41e3-94e2-84ac253f0c76-credential-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.202328 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.202144    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07d7fb36-40c1-41e3-94e2-84ac253f0c76-istio-podinfo\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:25.859850 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.859814    2563 generic.go:356] "Generic (PLEG): container finished" podID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerID="4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45" exitCode=0
Sep 14 08:16:25.860041 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.859861    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"
Sep 14 08:16:25.860041 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.859862    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" event={"ID":"07d7fb36-40c1-41e3-94e2-84ac253f0c76","Type":"ContainerDied","Data":"4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45"}
Sep 14 08:16:25.860041 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.859977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl" event={"ID":"07d7fb36-40c1-41e3-94e2-84ac253f0c76","Type":"ContainerDied","Data":"4ca18f35e903d74e80ec26eb1891647afc5203dc9c19a00281ee9b3b4dda7f68"}
Sep 14 08:16:25.860041 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.860008    2563 scope.go:117] "RemoveContainer" containerID="4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45"
Sep 14 08:16:25.873559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.873539    2563 scope.go:117] "RemoveContainer" containerID="f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"
Sep 14 08:16:25.882137 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.882120    2563 scope.go:117] "RemoveContainer" containerID="4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45"
Sep 14 08:16:25.882478 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:25.882449    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45\": container with ID starting with 4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45 not found: ID does not exist" containerID="4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45"
Sep 14 08:16:25.882565 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.882489    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45"} err="failed to get container status \"4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45\": rpc error: code = NotFound desc = could not find container \"4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45\": container with ID starting with 4facebd302fcb8bda42d2c7b728ea3a06b4ea409c5380989abafbb7e68f15a45 not found: ID does not exist"
Sep 14 08:16:25.882629 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.882567    2563 scope.go:117] "RemoveContainer" containerID="f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"
Sep 14 08:16:25.882936 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:25.882908    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592\": container with ID starting with f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592 not found: ID does not exist" containerID="f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"
Sep 14 08:16:25.883039 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.882943    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592"} err="failed to get container status \"f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592\": rpc error: code = NotFound desc = could not find container \"f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592\": container with ID starting with f76c8fb62d8dd6aec575e4fca65e187b5ea3852c2de590f90b4691fb3f514592 not found: ID does not exist"
Sep 14 08:16:25.885488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.885465    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"]
Sep 14 08:16:25.888652 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:25.888631    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b8346-openshift-default-589c895cb-jz8tl"]
Sep 14 08:16:26.789591 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:26.789558    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" path="/var/lib/kubelet/pods/07d7fb36-40c1-41e3-94e2-84ac253f0c76/volumes"
Sep 14 08:16:30.597681 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.597646    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"]
Sep 14 08:16:30.598133 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.597906    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g" podUID="8582672f-e002-4e1e-bfb2-74e6cdad92c6" containerName="maas-api" containerID="cri-o://c59a5911a5e726bcf05a704b919786a47bdae7c195442c20013a228d8e2db0b8" gracePeriod=30
Sep 14 08:16:30.627616 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.627589    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"]
Sep 14 08:16:30.630431 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.630405    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-4d8f28-29822895-7wn9k"]
Sep 14 08:16:30.655125 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.655091    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"]
Sep 14 08:16:30.655354 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.655329    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62" podUID="708a9c62-de0d-47b1-b56e-6137be60dd20" containerName="payload-processing" containerID="cri-o://c982162ddaf3005992e74db8d770effca2a356d1a9bf3b2f397c18a9e4159caf" gracePeriod=30
Sep 14 08:16:30.662872 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.662846    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"]
Sep 14 08:16:30.663150 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.663052    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4" podUID="d12ec295-3fcb-4d6d-8b5d-b8dae1449866" containerName="payload-pre-processing" containerID="cri-o://c225e79b5359fbb169cf9397e248fea65f572b547c2a592594a5114a4822e093" gracePeriod=30
Sep 14 08:16:30.719645 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:30.719611    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12ec295_3fcb_4d6d_8b5d_b8dae1449866.slice/crio-c225e79b5359fbb169cf9397e248fea65f572b547c2a592594a5114a4822e093.scope\": RecentStats: unable to find data in memory cache]"
Sep 14 08:16:30.792563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.792522    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b528e57a-c55b-4f51-a726-b51224b642e6" path="/var/lib/kubelet/pods/b528e57a-c55b-4f51-a726-b51224b642e6/volumes"
Sep 14 08:16:30.891143 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.891100    2563 generic.go:356] "Generic (PLEG): container finished" podID="d12ec295-3fcb-4d6d-8b5d-b8dae1449866" containerID="c225e79b5359fbb169cf9397e248fea65f572b547c2a592594a5114a4822e093" exitCode=0
Sep 14 08:16:30.891294 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.891137    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4" event={"ID":"d12ec295-3fcb-4d6d-8b5d-b8dae1449866","Type":"ContainerDied","Data":"c225e79b5359fbb169cf9397e248fea65f572b547c2a592594a5114a4822e093"}
Sep 14 08:16:30.893227 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.893204    2563 generic.go:356] "Generic (PLEG): container finished" podID="8582672f-e002-4e1e-bfb2-74e6cdad92c6" containerID="c59a5911a5e726bcf05a704b919786a47bdae7c195442c20013a228d8e2db0b8" exitCode=0
Sep 14 08:16:30.893358 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.893287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g" event={"ID":"8582672f-e002-4e1e-bfb2-74e6cdad92c6","Type":"ContainerDied","Data":"c59a5911a5e726bcf05a704b919786a47bdae7c195442c20013a228d8e2db0b8"}
Sep 14 08:16:30.895040 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.895017    2563 generic.go:356] "Generic (PLEG): container finished" podID="708a9c62-de0d-47b1-b56e-6137be60dd20" containerID="c982162ddaf3005992e74db8d770effca2a356d1a9bf3b2f397c18a9e4159caf" exitCode=0
Sep 14 08:16:30.895151 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.895062    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62" event={"ID":"708a9c62-de0d-47b1-b56e-6137be60dd20","Type":"ContainerDied","Data":"c982162ddaf3005992e74db8d770effca2a356d1a9bf3b2f397c18a9e4159caf"}
Sep 14 08:16:30.920348 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.920324    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:16:30.953916 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.953888    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:16:30.963185 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:30.963168    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:16:31.056650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056618    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-metrics-certs\") pod \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") "
Sep 14 08:16:31.056807 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056673    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppptc\" (UniqueName: \"kubernetes.io/projected/708a9c62-de0d-47b1-b56e-6137be60dd20-kube-api-access-ppptc\") pod \"708a9c62-de0d-47b1-b56e-6137be60dd20\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") "
Sep 14 08:16:31.056874 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056838    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmrcs\" (UniqueName: \"kubernetes.io/projected/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-kube-api-access-mmrcs\") pod \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") "
Sep 14 08:16:31.056930 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056890    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr7j2\" (UniqueName: \"kubernetes.io/projected/8582672f-e002-4e1e-bfb2-74e6cdad92c6-kube-api-access-nr7j2\") pod \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") "
Sep 14 08:16:31.056930 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-plugins-config-volume\") pod \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\" (UID: \"d12ec295-3fcb-4d6d-8b5d-b8dae1449866\") "
Sep 14 08:16:31.057048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056962    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8582672f-e002-4e1e-bfb2-74e6cdad92c6-openshift-service-ca\") pod \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") "
Sep 14 08:16:31.057048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.056986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-maas-api-tls\") pod \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\" (UID: \"8582672f-e002-4e1e-bfb2-74e6cdad92c6\") "
Sep 14 08:16:31.057698 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.057232    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708a9c62-de0d-47b1-b56e-6137be60dd20-plugins-config-volume\") pod \"708a9c62-de0d-47b1-b56e-6137be60dd20\" (UID: \"708a9c62-de0d-47b1-b56e-6137be60dd20\") "
Sep 14 08:16:31.057698 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.057341    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d12ec295-3fcb-4d6d-8b5d-b8dae1449866" (UID: "d12ec295-3fcb-4d6d-8b5d-b8dae1449866"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:31.057698 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.057376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8582672f-e002-4e1e-bfb2-74e6cdad92c6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8582672f-e002-4e1e-bfb2-74e6cdad92c6" (UID: "8582672f-e002-4e1e-bfb2-74e6cdad92c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:31.057698 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.057645    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.057698 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.057669    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8582672f-e002-4e1e-bfb2-74e6cdad92c6-openshift-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.058066 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.057783    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/708a9c62-de0d-47b1-b56e-6137be60dd20-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "708a9c62-de0d-47b1-b56e-6137be60dd20" (UID: "708a9c62-de0d-47b1-b56e-6137be60dd20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:16:31.059079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.059042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-kube-api-access-mmrcs" (OuterVolumeSpecName: "kube-api-access-mmrcs") pod "d12ec295-3fcb-4d6d-8b5d-b8dae1449866" (UID: "d12ec295-3fcb-4d6d-8b5d-b8dae1449866"). InnerVolumeSpecName "kube-api-access-mmrcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:31.059079 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.059060    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708a9c62-de0d-47b1-b56e-6137be60dd20-kube-api-access-ppptc" (OuterVolumeSpecName: "kube-api-access-ppptc") pod "708a9c62-de0d-47b1-b56e-6137be60dd20" (UID: "708a9c62-de0d-47b1-b56e-6137be60dd20"). InnerVolumeSpecName "kube-api-access-ppptc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:31.059236 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.059145    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8582672f-e002-4e1e-bfb2-74e6cdad92c6" (UID: "8582672f-e002-4e1e-bfb2-74e6cdad92c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:31.059236 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.059201    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8582672f-e002-4e1e-bfb2-74e6cdad92c6-kube-api-access-nr7j2" (OuterVolumeSpecName: "kube-api-access-nr7j2") pod "8582672f-e002-4e1e-bfb2-74e6cdad92c6" (UID: "8582672f-e002-4e1e-bfb2-74e6cdad92c6"). InnerVolumeSpecName "kube-api-access-nr7j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:31.059407 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.059388    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8582672f-e002-4e1e-bfb2-74e6cdad92c6" (UID: "8582672f-e002-4e1e-bfb2-74e6cdad92c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:31.159042 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.158975    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-maas-api-tls\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.159042 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.159008    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708a9c62-de0d-47b1-b56e-6137be60dd20-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.159042 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.159023    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8582672f-e002-4e1e-bfb2-74e6cdad92c6-metrics-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.159042 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.159039    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppptc\" (UniqueName: \"kubernetes.io/projected/708a9c62-de0d-47b1-b56e-6137be60dd20-kube-api-access-ppptc\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.159248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.159054    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmrcs\" (UniqueName: \"kubernetes.io/projected/d12ec295-3fcb-4d6d-8b5d-b8dae1449866-kube-api-access-mmrcs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.159248 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.159068    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr7j2\" (UniqueName: \"kubernetes.io/projected/8582672f-e002-4e1e-bfb2-74e6cdad92c6-kube-api-access-nr7j2\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:16:31.902220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.902137    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"
Sep 14 08:16:31.902220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.902172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62" event={"ID":"708a9c62-de0d-47b1-b56e-6137be60dd20","Type":"ContainerDied","Data":"ab668744b0dc0d99098a5a40f9b8066022cffbaef8b87946c2fe182b9f3b1004"}
Sep 14 08:16:31.902701 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.902253    2563 scope.go:117] "RemoveContainer" containerID="c982162ddaf3005992e74db8d770effca2a356d1a9bf3b2f397c18a9e4159caf"
Sep 14 08:16:31.903630 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.903615    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"
Sep 14 08:16:31.903699 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.903667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4" event={"ID":"d12ec295-3fcb-4d6d-8b5d-b8dae1449866","Type":"ContainerDied","Data":"af7efe43f2b7b6e0e268c2ab14ddfd17a4d3e7ec596d04f625531318d51a36cb"}
Sep 14 08:16:31.905400 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.905368    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"
Sep 14 08:16:31.905498 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.905366    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g" event={"ID":"8582672f-e002-4e1e-bfb2-74e6cdad92c6","Type":"ContainerDied","Data":"3f606950e5438e0c1576849cff59b2b741b0187bc6e34b8151a0b4d8af583c61"}
Sep 14 08:16:31.913154 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.913134    2563 scope.go:117] "RemoveContainer" containerID="c225e79b5359fbb169cf9397e248fea65f572b547c2a592594a5114a4822e093"
Sep 14 08:16:31.921896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.921874    2563 scope.go:117] "RemoveContainer" containerID="c59a5911a5e726bcf05a704b919786a47bdae7c195442c20013a228d8e2db0b8"
Sep 14 08:16:31.936685 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.936652    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"]
Sep 14 08:16:31.939756 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.939675    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-4d8f28-577758cdf5-hmj62"]
Sep 14 08:16:31.943081 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.943056    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"]
Sep 14 08:16:31.948709 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.948687    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4d8f28-846c85bcfd-n5ks4"]
Sep 14 08:16:31.957802 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.957779    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"]
Sep 14 08:16:31.962225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:31.962204    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-4d8f28-994cd8b96-9hx7g"]
Sep 14 08:16:32.789038 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:32.789007    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="708a9c62-de0d-47b1-b56e-6137be60dd20" path="/var/lib/kubelet/pods/708a9c62-de0d-47b1-b56e-6137be60dd20/volumes"
Sep 14 08:16:32.789379 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:32.789367    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8582672f-e002-4e1e-bfb2-74e6cdad92c6" path="/var/lib/kubelet/pods/8582672f-e002-4e1e-bfb2-74e6cdad92c6/volumes"
Sep 14 08:16:32.789683 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:32.789671    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d12ec295-3fcb-4d6d-8b5d-b8dae1449866" path="/var/lib/kubelet/pods/d12ec295-3fcb-4d6d-8b5d-b8dae1449866/volumes"
Sep 14 08:16:42.836141 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:42.836098    2563 pod_container_manager_linux.go:217] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","pod3a3dcf97-eb10-4dc1-92a8-497fd83363ea"] err="unable to destroy cgroup paths for cgroup [kubepods burstable pod3a3dcf97-eb10-4dc1-92a8-497fd83363ea] : Timed out while waiting for systemd to remove kubepods-burstable-pod3a3dcf97_eb10_4dc1_92a8_497fd83363ea.slice"
Sep 14 08:16:42.836595 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:16:42.836143    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to delete cgroup paths for [kubepods burstable pod3a3dcf97-eb10-4dc1-92a8-497fd83363ea] : unable to destroy cgroup paths for cgroup [kubepods burstable pod3a3dcf97-eb10-4dc1-92a8-497fd83363ea] : Timed out while waiting for systemd to remove kubepods-burstable-pod3a3dcf97_eb10_4dc1_92a8_497fd83363ea.slice" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx" podUID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea"
Sep 14 08:16:42.966794 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:42.966764    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"
Sep 14 08:16:42.986014 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:42.985972    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"]
Sep 14 08:16:42.990195 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:42.990170    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b8346-5d9676ff6d-w6mqx"]
Sep 14 08:16:44.789753 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:16:44.789708    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea" path="/var/lib/kubelet/pods/3a3dcf97-eb10-4dc1-92a8-497fd83363ea/volumes"
Sep 14 08:17:59.485700 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.485666    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-1fd46d"]
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486073    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cabb284-31b5-4d4e-8da8-936cacba1324" containerName="revoke-keys"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486085    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cabb284-31b5-4d4e-8da8-936cacba1324" containerName="revoke-keys"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486097    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486103    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486116    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="433ac3d3-d9df-40f2-b40c-9c902d535f79" containerName="main"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486122    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="433ac3d3-d9df-40f2-b40c-9c902d535f79" containerName="main"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486129    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8540d7c2-55c1-4e92-b260-b00a2ae89591" containerName="main"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486135    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8540d7c2-55c1-4e92-b260-b00a2ae89591" containerName="main"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486144    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eaa106f-9457-45f8-b699-a2c95aadb1ae" containerName="payload-pre-processing"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486149    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eaa106f-9457-45f8-b699-a2c95aadb1ae" containerName="payload-pre-processing"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486156    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ea5649-56da-4ebf-9515-da506c65cd74" containerName="payload-pre-processing"
Sep 14 08:17:59.486157 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486162    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ea5649-56da-4ebf-9515-da506c65cd74" containerName="payload-pre-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486175    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea" containerName="maas-api"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486180    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea" containerName="maas-api"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486189    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8582672f-e002-4e1e-bfb2-74e6cdad92c6" containerName="maas-api"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486194    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8582672f-e002-4e1e-bfb2-74e6cdad92c6" containerName="maas-api"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486201    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="669ebfb4-dc18-4ef5-99c4-0d626661bff7" containerName="payload-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486207    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="669ebfb4-dc18-4ef5-99c4-0d626661bff7" containerName="payload-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486214    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d12ec295-3fcb-4d6d-8b5d-b8dae1449866" containerName="payload-pre-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486219    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d12ec295-3fcb-4d6d-8b5d-b8dae1449866" containerName="payload-pre-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486227    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="708a9c62-de0d-47b1-b56e-6137be60dd20" containerName="payload-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486232    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="708a9c62-de0d-47b1-b56e-6137be60dd20" containerName="payload-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486286    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8540d7c2-55c1-4e92-b260-b00a2ae89591" containerName="main"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486301    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="669ebfb4-dc18-4ef5-99c4-0d626661bff7" containerName="payload-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486309    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486314    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cabb284-31b5-4d4e-8da8-936cacba1324" containerName="revoke-keys"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486320    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d12ec295-3fcb-4d6d-8b5d-b8dae1449866" containerName="payload-pre-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486328    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a3dcf97-eb10-4dc1-92a8-497fd83363ea" containerName="maas-api"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486334    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="708a9c62-de0d-47b1-b56e-6137be60dd20" containerName="payload-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486340    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ea5649-56da-4ebf-9515-da506c65cd74" containerName="payload-pre-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486345    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="433ac3d3-d9df-40f2-b40c-9c902d535f79" containerName="main"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486352    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eaa106f-9457-45f8-b699-a2c95aadb1ae" containerName="payload-pre-processing"
Sep 14 08:17:59.486542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.486358    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8582672f-e002-4e1e-bfb2-74e6cdad92c6" containerName="maas-api"
Sep 14 08:17:59.489606 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.489590    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:17:59.492564 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.492540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qcf9c\""
Sep 14 08:17:59.494357 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.494331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-1fd46d"]
Sep 14 08:17:59.560306 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.560275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj7b2\" (UniqueName: \"kubernetes.io/projected/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0-kube-api-access-wj7b2\") pod \"test-curl-7752-1fd46d\" (UID: \"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0\") " pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:17:59.661772 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.661708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj7b2\" (UniqueName: \"kubernetes.io/projected/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0-kube-api-access-wj7b2\") pod \"test-curl-7752-1fd46d\" (UID: \"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0\") " pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:17:59.669441 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.669408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj7b2\" (UniqueName: \"kubernetes.io/projected/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0-kube-api-access-wj7b2\") pod \"test-curl-7752-1fd46d\" (UID: \"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0\") " pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:17:59.800680 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:17:59.800603    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:18:00.071505 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:00.071473    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-1fd46d"]
Sep 14 08:18:00.074397 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:18:00.074365    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50 WatchSource:0}: Error finding container e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50: Status 404 returned error can't find the container with id e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50
Sep 14 08:18:00.389635 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:00.389535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-1fd46d" event={"ID":"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0","Type":"ContainerStarted","Data":"37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567"}
Sep 14 08:18:00.389635 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:00.389585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-1fd46d" event={"ID":"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0","Type":"ContainerStarted","Data":"e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50"}
Sep 14 08:18:00.405148 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:00.405096    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-1fd46d" podStartSLOduration=1.405080587 podStartE2EDuration="1.405080587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:00.402504258 +0000 UTC m=+1736.110518410" watchObservedRunningTime="2026-09-14 08:18:00.405080587 +0000 UTC m=+1736.113094775"
Sep 14 08:18:01.018373 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.018340    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-1fd46d"]
Sep 14 08:18:01.021764 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.021738    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-1fd46d"]
Sep 14 08:18:01.142341 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142248    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7752-608710"]
Sep 14 08:18:01.142751 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" containerName="test-curl-7752-1fd46d"
Sep 14 08:18:01.142835 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142752    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" containerName="test-curl-7752-1fd46d"
Sep 14 08:18:01.142835 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142772    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy"
Sep 14 08:18:01.142835 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142778    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy"
Sep 14 08:18:01.142996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142852    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" containerName="test-curl-7752-1fd46d"
Sep 14 08:18:01.142996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.142861    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d7fb36-40c1-41e3-94e2-84ac253f0c76" containerName="istio-proxy"
Sep 14 08:18:01.146093 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.146074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:01.151347 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.151326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-608710"]
Sep 14 08:18:01.278331 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.278287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsg8g\" (UniqueName: \"kubernetes.io/projected/8db48997-cf19-4975-9d0d-6d489d8bcb44-kube-api-access-wsg8g\") pod \"test-curl-7752-608710\" (UID: \"8db48997-cf19-4975-9d0d-6d489d8bcb44\") " pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:01.379614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.379582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsg8g\" (UniqueName: \"kubernetes.io/projected/8db48997-cf19-4975-9d0d-6d489d8bcb44-kube-api-access-wsg8g\") pod \"test-curl-7752-608710\" (UID: \"8db48997-cf19-4975-9d0d-6d489d8bcb44\") " pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:01.387453 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.387427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsg8g\" (UniqueName: \"kubernetes.io/projected/8db48997-cf19-4975-9d0d-6d489d8bcb44-kube-api-access-wsg8g\") pod \"test-curl-7752-608710\" (UID: \"8db48997-cf19-4975-9d0d-6d489d8bcb44\") " pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:01.394924 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:01.394864    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-1fd46d" container="test-curl-7752-1fd46d"
Sep 14 08:18:01.457236 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.457195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:01.726808 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:01.726781    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7752-608710"]
Sep 14 08:18:01.729086 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:18:01.729043    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8db48997_cf19_4975_9d0d_6d489d8bcb44.slice/crio-5c36bbcf326a0e43f8fee418baca691c25528dfdf021976c35ab46ab36eb2d6b WatchSource:0}: Error finding container 5c36bbcf326a0e43f8fee418baca691c25528dfdf021976c35ab46ab36eb2d6b: Status 404 returned error can't find the container with id 5c36bbcf326a0e43f8fee418baca691c25528dfdf021976c35ab46ab36eb2d6b
Sep 14 08:18:02.401423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:02.401379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-608710" event={"ID":"8db48997-cf19-4975-9d0d-6d489d8bcb44","Type":"ContainerStarted","Data":"43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4"}
Sep 14 08:18:02.401423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:02.401424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7752-608710" event={"ID":"8db48997-cf19-4975-9d0d-6d489d8bcb44","Type":"ContainerStarted","Data":"5c36bbcf326a0e43f8fee418baca691c25528dfdf021976c35ab46ab36eb2d6b"}
Sep 14 08:18:02.401894 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:02.401447    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-1fd46d" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" containerName="test-curl-7752-1fd46d" containerID="cri-o://37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567" gracePeriod=2
Sep 14 08:18:02.403370 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:02.403344    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-1fd46d\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:18:02.417934 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:02.417892    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7752-608710" podStartSLOduration=1.417879294 podStartE2EDuration="1.417879294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:02.416965123 +0000 UTC m=+1738.124979265" watchObservedRunningTime="2026-09-14 08:18:02.417879294 +0000 UTC m=+1738.125893434"
Sep 14 08:18:02.418888 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:02.418863    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-1fd46d\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:18:03.010155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:03.010119    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7752-608710"]
Sep 14 08:18:03.012455 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:03.012427    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7752-608710"]
Sep 14 08:18:03.406945 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:03.406915    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7752-608710" container="test-curl-7752-608710"
Sep 14 08:18:04.412420 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:04.412364    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7752-608710" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" containerName="test-curl-7752-608710" containerID="cri-o://43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4" gracePeriod=2
Sep 14 08:18:04.414515 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:04.414482    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-608710\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:04.547841 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:04.547821    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:18:04.550583 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:04.550556    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-608710\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:04.710220 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:04.710192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj7b2\" (UniqueName: \"kubernetes.io/projected/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0-kube-api-access-wj7b2\") pod \"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0\" (UID: \"f84d0673-5dda-4dbd-8b1b-e8d2f38902e0\") "
Sep 14 08:18:04.712249 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:04.712223    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0-kube-api-access-wj7b2" (OuterVolumeSpecName: "kube-api-access-wj7b2") pod "f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" (UID: "f84d0673-5dda-4dbd-8b1b-e8d2f38902e0"). InnerVolumeSpecName "kube-api-access-wj7b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:04.788511 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:04.788471    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7752-608710\" is forbidden: User \"system:node:ip-10-0-135-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-172.ec2.internal' and this object" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:04.790377 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:04.790354    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" path="/var/lib/kubelet/pods/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0/volumes"
Sep 14 08:18:04.810831 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:04.810806    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj7b2\" (UniqueName: \"kubernetes.io/projected/f84d0673-5dda-4dbd-8b1b-e8d2f38902e0-kube-api-access-wj7b2\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:18:05.162764 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:05.162665    2563 scope.go:117] "RemoveContainer" containerID="4c537c096dae11423c6571341a34a0dd5c5d8c15ef1b4cabe01774235d2a6fd9"
Sep 14 08:18:05.418782 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:05.418674    2563 generic.go:356] "Generic (PLEG): container finished" podID="f84d0673-5dda-4dbd-8b1b-e8d2f38902e0" containerID="37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567" exitCode=137
Sep 14 08:18:05.418782 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:05.418717    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-1fd46d"
Sep 14 08:18:05.419207 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:05.418781    2563 scope.go:117] "RemoveContainer" containerID="37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567"
Sep 14 08:18:05.427094 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:05.427078    2563 scope.go:117] "RemoveContainer" containerID="37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567"
Sep 14 08:18:05.427375 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:05.427357    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567\": container with ID starting with 37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567 not found: ID does not exist" containerID="37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567"
Sep 14 08:18:05.427431 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:05.427383    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567"} err="failed to get container status \"37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567\": rpc error: code = NotFound desc = could not find container \"37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567\": container with ID starting with 37953df40fbfaaaf26477aa6409d04524129123c129c9b13cd88621b65bdf567 not found: ID does not exist"
Sep 14 08:18:06.553360 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:06.553333    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:06.624675 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:06.624648    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsg8g\" (UniqueName: \"kubernetes.io/projected/8db48997-cf19-4975-9d0d-6d489d8bcb44-kube-api-access-wsg8g\") pod \"8db48997-cf19-4975-9d0d-6d489d8bcb44\" (UID: \"8db48997-cf19-4975-9d0d-6d489d8bcb44\") "
Sep 14 08:18:06.626766 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:06.626702    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db48997-cf19-4975-9d0d-6d489d8bcb44-kube-api-access-wsg8g" (OuterVolumeSpecName: "kube-api-access-wsg8g") pod "8db48997-cf19-4975-9d0d-6d489d8bcb44" (UID: "8db48997-cf19-4975-9d0d-6d489d8bcb44"). InnerVolumeSpecName "kube-api-access-wsg8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:06.725559 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:06.725535    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsg8g\" (UniqueName: \"kubernetes.io/projected/8db48997-cf19-4975-9d0d-6d489d8bcb44-kube-api-access-wsg8g\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:18:06.789198 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:06.789171    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" path="/var/lib/kubelet/pods/8db48997-cf19-4975-9d0d-6d489d8bcb44/volumes"
Sep 14 08:18:07.432170 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:07.432133    2563 generic.go:356] "Generic (PLEG): container finished" podID="8db48997-cf19-4975-9d0d-6d489d8bcb44" containerID="43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4" exitCode=137
Sep 14 08:18:07.432343 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:07.432177    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7752-608710"
Sep 14 08:18:07.432343 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:07.432234    2563 scope.go:117] "RemoveContainer" containerID="43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4"
Sep 14 08:18:07.441225 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:07.441206    2563 scope.go:117] "RemoveContainer" containerID="43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4"
Sep 14 08:18:07.441539 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:07.441521    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4\": container with ID starting with 43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4 not found: ID does not exist" containerID="43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4"
Sep 14 08:18:07.441613 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:18:07.441543    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4"} err="failed to get container status \"43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4\": rpc error: code = NotFound desc = could not find container \"43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4\": container with ID starting with 43cbdf5959ed9c167fe5061142f74029c8dbac82034574b9bc64c151375143c4 not found: ID does not exist"
Sep 14 08:18:07.800614 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:07.800532    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:09.027214 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:09.027170    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:09.027647 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:09.027179    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:10.994219 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:10.994185    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:21.048656 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:21.048616    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:22.765077 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:22.765040    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:31.103588 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:31.103555    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:37.794143 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:37.794106    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:41.113391 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:41.113356    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:51.122963 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:51.122926    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:18:52.768441 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:18:52.768404    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:19:01.176027 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:19:01.175945    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf84d0673_5dda_4dbd_8b1b_e8d2f38902e0.slice/crio-e152d8acee6db1ccb8fd8b16dec33eb92c5885c1e8df6ec16038ba07665cae50\": RecentStats: unable to find data in memory cache]"
Sep 14 08:19:04.852007 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:04.851905    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:19:04.859940 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:04.856330    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:19:21.575987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.575948    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"]
Sep 14 08:19:21.576563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.576545    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" containerName="test-curl-7752-608710"
Sep 14 08:19:21.576612 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.576565    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" containerName="test-curl-7752-608710"
Sep 14 08:19:21.576650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.576645    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db48997-cf19-4975-9d0d-6d489d8bcb44" containerName="test-curl-7752-608710"
Sep 14 08:19:21.579839 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.579820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.582232 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.582211    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 08:19:21.582333 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.582211    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-03d6fa-openshift-default-dockercfg-nq58j\""
Sep 14 08:19:21.589922 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.589898    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"]
Sep 14 08:19:21.677478 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21431a6-a018-444c-b613-6f7e3a952667-istiod-ca-cert\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21431a6-a018-444c-b613-6f7e3a952667-istio-podinfo\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-istio-token\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-socket\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-data\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-credential-socket\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-envoy\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.677650 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6k9h\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-kube-api-access-h6k9h\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.678009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.677685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-certs\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778184 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21431a6-a018-444c-b613-6f7e3a952667-istiod-ca-cert\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778184 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21431a6-a018-444c-b613-6f7e3a952667-istio-podinfo\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-istio-token\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-socket\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-data\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778581 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-credential-socket\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778581 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-envoy\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778581 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6k9h\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-kube-api-access-h6k9h\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778581 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-certs\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778814 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-socket\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778814 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778709    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-data\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778814 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-credential-socket\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.778970 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.778922    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-certs\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.779183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.779163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21431a6-a018-444c-b613-6f7e3a952667-istiod-ca-cert\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.780639 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.780619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-envoy\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.780797 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.780781    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21431a6-a018-444c-b613-6f7e3a952667-istio-podinfo\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.786535 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.786507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6k9h\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-kube-api-access-h6k9h\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.786535 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.786528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-istio-token\") pod \"e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") " pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:21.892281 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:21.892196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:22.166495 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.166469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"]
Sep 14 08:19:22.168664 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:19:22.168632    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21431a6_a018_444c_b613_6f7e3a952667.slice/crio-cc79b05ae7c231812d2bb819f510f6f5772e0dfca7c3567e55b53e4ca4cd1a96 WatchSource:0}: Error finding container cc79b05ae7c231812d2bb819f510f6f5772e0dfca7c3567e55b53e4ca4cd1a96: Status 404 returned error can't find the container with id cc79b05ae7c231812d2bb819f510f6f5772e0dfca7c3567e55b53e4ca4cd1a96
Sep 14 08:19:22.171146 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.171105    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:19:22.171238 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.171200    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:19:22.171282 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.171251    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:19:22.837990 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.837947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" event={"ID":"d21431a6-a018-444c-b613-6f7e3a952667","Type":"ContainerStarted","Data":"6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"}
Sep 14 08:19:22.838446 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.838004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" event={"ID":"d21431a6-a018-444c-b613-6f7e3a952667","Type":"ContainerStarted","Data":"cc79b05ae7c231812d2bb819f510f6f5772e0dfca7c3567e55b53e4ca4cd1a96"}
Sep 14 08:19:22.858331 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.858279    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podStartSLOduration=1.85826401 podStartE2EDuration="1.85826401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:22.857380548 +0000 UTC m=+1818.565394687" watchObservedRunningTime="2026-09-14 08:19:22.85826401 +0000 UTC m=+1818.566278148"
Sep 14 08:19:22.892829 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:22.892807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:23.898044 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:23.898011    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:24.849048 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:24.849016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:24.850080 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:24.850056    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:40.774183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.774146    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": read tcp 10.133.0.2:42348->10.133.0.97:15021: read: connection reset by peer" start-of-body=
Sep 14 08:19:40.774558 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.774210    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": read tcp 10.133.0.2:42348->10.133.0.97:15021: read: connection reset by peer"
Sep 14 08:19:40.933127 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.933096    2563 generic.go:356] "Generic (PLEG): container finished" podID="d21431a6-a018-444c-b613-6f7e3a952667" containerID="6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706" exitCode=0
Sep 14 08:19:40.933296 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.933135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" event={"ID":"d21431a6-a018-444c-b613-6f7e3a952667","Type":"ContainerDied","Data":"6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"}
Sep 14 08:19:40.933548 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.933533    2563 scope.go:117] "RemoveContainer" containerID="6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"
Sep 14 08:19:40.934688 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.934659    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:19:40.934768 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.934754    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:19:40.934814 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:40.934794    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:19:41.942692 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:41.941751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" event={"ID":"d21431a6-a018-444c-b613-6f7e3a952667","Type":"ContainerStarted","Data":"4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874"}
Sep 14 08:19:42.892492 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:42.892458    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:42.892754 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:42.892708    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Sep 14 08:19:42.892861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:42.892785    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Sep 14 08:19:43.893167 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:43.893133    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Sep 14 08:19:43.893589 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:43.893190    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Sep 14 08:19:44.893306 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:44.893273    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Sep 14 08:19:44.893703 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:44.893327    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Sep 14 08:19:45.893520 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:45.893476    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Sep 14 08:19:45.893934 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:45.893552    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Sep 14 08:19:46.898287 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:46.898250    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:46.898765 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:46.898663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:46.899336 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:46.899307    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:19:56.419582 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.419541    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"]
Sep 14 08:19:56.423426 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.423403    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.425982 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.425961    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-7bcf7b\""
Sep 14 08:19:56.426084 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.426048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-7bcf7b-dockercfg-skcdx\""
Sep 14 08:19:56.431789 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.431768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"]
Sep 14 08:19:56.522925 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.522893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8whx\" (UniqueName: \"kubernetes.io/projected/74fb50b0-6409-4ffa-a3d4-05daf19492e9-kube-api-access-s8whx\") pod \"payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.523106 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.523083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74fb50b0-6409-4ffa-a3d4-05daf19492e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.623956 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.623924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74fb50b0-6409-4ffa-a3d4-05daf19492e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.624094 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.623970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8whx\" (UniqueName: \"kubernetes.io/projected/74fb50b0-6409-4ffa-a3d4-05daf19492e9-kube-api-access-s8whx\") pod \"payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.624603 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.624585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74fb50b0-6409-4ffa-a3d4-05daf19492e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.631515 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.631490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8whx\" (UniqueName: \"kubernetes.io/projected/74fb50b0-6409-4ffa-a3d4-05daf19492e9-kube-api-access-s8whx\") pod \"payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:56.734496 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:56.734468    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:57.013475 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:57.013393    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"]
Sep 14 08:19:57.017178 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:19:57.017151    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fb50b0_6409_4ffa_a3d4_05daf19492e9.slice/crio-2e13641e3a3af795881d9c9f889f9dc2344892e065904fff986f9df644691a63 WatchSource:0}: Error finding container 2e13641e3a3af795881d9c9f889f9dc2344892e065904fff986f9df644691a63: Status 404 returned error can't find the container with id 2e13641e3a3af795881d9c9f889f9dc2344892e065904fff986f9df644691a63
Sep 14 08:19:57.033091 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:57.033061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp" event={"ID":"74fb50b0-6409-4ffa-a3d4-05daf19492e9","Type":"ContainerStarted","Data":"2e13641e3a3af795881d9c9f889f9dc2344892e065904fff986f9df644691a63"}
Sep 14 08:19:58.040043 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:58.040002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp" event={"ID":"74fb50b0-6409-4ffa-a3d4-05daf19492e9","Type":"ContainerStarted","Data":"533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f"}
Sep 14 08:19:58.040508 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:58.040118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:19:58.056448 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:19:58.056403    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp" podStartSLOduration=2.056387705 podStartE2EDuration="2.056387705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:19:58.05497128 +0000 UTC m=+1853.762985434" watchObservedRunningTime="2026-09-14 08:19:58.056387705 +0000 UTC m=+1853.764401844"
Sep 14 08:20:09.049241 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:09.049208    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:20:32.693092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.692994    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"]
Sep 14 08:20:32.696948 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.696924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.699347 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.699326    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-03d6fa\"/\"kube-root-ca.crt\""
Sep 14 08:20:32.699464 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.699445    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-03d6fa\"/\"default-dockercfg-vhwqn\""
Sep 14 08:20:32.700393 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.700344    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-03d6fa\"/\"openshift-service-ca.crt\""
Sep 14 08:20:32.700492 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.700477    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-03d6fa\"/\"ipp-route-03d6fa-kserve-self-signed-certs\""
Sep 14 08:20:32.715642 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.715616    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"]
Sep 14 08:20:32.866669 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.866635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tls-certs\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.866846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.866679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-home\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.866846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.866720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-dshm\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.866846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.866789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tmp-dir\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.867023 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.866862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrbz\" (UniqueName: \"kubernetes.io/projected/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-kube-api-access-8wrbz\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.867023 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.866959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-model-cache\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.968580 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.968496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tls-certs\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.968580 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.968566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-home\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.968842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.968630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-dshm\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.968842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.968656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tmp-dir\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.968842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.968692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrbz\" (UniqueName: \"kubernetes.io/projected/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-kube-api-access-8wrbz\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.968842 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.968801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-model-cache\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.969114 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.969090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tmp-dir\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.969162 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.969127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-model-cache\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.969288 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.969266    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-home\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.970932 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.970901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-dshm\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.971147 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.971128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tls-certs\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:32.976977 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:32.976957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrbz\" (UniqueName: \"kubernetes.io/projected/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-kube-api-access-8wrbz\") pod \"ipp-route-03d6fa-kserve-56965b9cf-466s7\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") " pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:33.016463 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:33.016444    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:33.289571 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:33.289546    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"]
Sep 14 08:20:33.291074 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:20:33.291049    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17cf0e76_60b7_417b_ae8b_2b85a0f66c07.slice/crio-20abbc28f1eb31fdf329c9849b678bef6aae55a00b04f69d1c8c3f434343ac5b WatchSource:0}: Error finding container 20abbc28f1eb31fdf329c9849b678bef6aae55a00b04f69d1c8c3f434343ac5b: Status 404 returned error can't find the container with id 20abbc28f1eb31fdf329c9849b678bef6aae55a00b04f69d1c8c3f434343ac5b
Sep 14 08:20:34.236955 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:34.236920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7" event={"ID":"17cf0e76-60b7-417b-ae8b-2b85a0f66c07","Type":"ContainerStarted","Data":"e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17"}
Sep 14 08:20:34.236955 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:34.236969    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7" event={"ID":"17cf0e76-60b7-417b-ae8b-2b85a0f66c07","Type":"ContainerStarted","Data":"20abbc28f1eb31fdf329c9849b678bef6aae55a00b04f69d1c8c3f434343ac5b"}
Sep 14 08:20:34.253411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:34.253364    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7" podStartSLOduration=2.253350462 podStartE2EDuration="2.253350462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:20:34.252894354 +0000 UTC m=+1889.960908497" watchObservedRunningTime="2026-09-14 08:20:34.253350462 +0000 UTC m=+1889.961364602"
Sep 14 08:20:43.017464 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:43.017427    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:43.017909 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:43.017486    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:43.030072 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:43.030043    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:20:43.297022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:20:43.296945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:21:05.208548 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:05.208517    2563 scope.go:117] "RemoveContainer" containerID="4483f9a801c4610dd25a4118ed2c431841ee6fec16b0e9ac3a69167396eaf895"
Sep 14 08:21:05.217263 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:05.217243    2563 scope.go:117] "RemoveContainer" containerID="b224c82cfc1f3323ebacdffd70733118fe79331ac7e10698021f011ec80b074c"
Sep 14 08:21:05.225098 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:05.225075    2563 scope.go:117] "RemoveContainer" containerID="80028144375cd95cf0a4392223bd102c2178d2bf5a0b4b36378c51315f96c0dd"
Sep 14 08:21:12.622671 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.622631    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"]
Sep 14 08:21:12.626504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.626479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.629163 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.629144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-70893f-dockercfg-ncjzl\""
Sep 14 08:21:12.629281 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.629216    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-70893f\""
Sep 14 08:21:12.634706 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.634684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"]
Sep 14 08:21:12.668540 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.668516    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"]
Sep 14 08:21:12.672093 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.672077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.682027 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.682006    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"]
Sep 14 08:21:12.745496 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.745458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f642n\" (UniqueName: \"kubernetes.io/projected/68464c35-0bb2-4961-aebc-36335c72a0f9-kube-api-access-f642n\") pod \"payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.745667 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.745520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68464c35-0bb2-4961-aebc-36335c72a0f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.846964 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.846934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsd98\" (UniqueName: \"kubernetes.io/projected/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-kube-api-access-fsd98\") pod \"payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.847128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.846985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f642n\" (UniqueName: \"kubernetes.io/projected/68464c35-0bb2-4961-aebc-36335c72a0f9-kube-api-access-f642n\") pod \"payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.847128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.847043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68464c35-0bb2-4961-aebc-36335c72a0f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.847250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.847155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.847817 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.847795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68464c35-0bb2-4961-aebc-36335c72a0f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.856206 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.856184    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f642n\" (UniqueName: \"kubernetes.io/projected/68464c35-0bb2-4961-aebc-36335c72a0f9-kube-api-access-f642n\") pod \"payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.938861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.938833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:12.947906 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.947886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.947996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.947927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsd98\" (UniqueName: \"kubernetes.io/projected/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-kube-api-access-fsd98\") pod \"payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.948609 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.948588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.955979 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.955956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsd98\" (UniqueName: \"kubernetes.io/projected/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-kube-api-access-fsd98\") pod \"payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:12.982966 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:12.982935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:13.214067 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.214043    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"]
Sep 14 08:21:13.215700 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:21:13.215672    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68464c35_0bb2_4961_aebc_36335c72a0f9.slice/crio-8cc715775784a949b18364c4c169c0314132f55a24145490ecaba8373e814750 WatchSource:0}: Error finding container 8cc715775784a949b18364c4c169c0314132f55a24145490ecaba8373e814750: Status 404 returned error can't find the container with id 8cc715775784a949b18364c4c169c0314132f55a24145490ecaba8373e814750
Sep 14 08:21:13.273782 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.273758    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"]
Sep 14 08:21:13.276709 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:21:13.276688    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00aca3d7_aec8_4f0d_8c13_7d00ed43678b.slice/crio-522fbed8db5eb67e75e17ef5498b523331533f608ae1b201126b0804740136de WatchSource:0}: Error finding container 522fbed8db5eb67e75e17ef5498b523331533f608ae1b201126b0804740136de: Status 404 returned error can't find the container with id 522fbed8db5eb67e75e17ef5498b523331533f608ae1b201126b0804740136de
Sep 14 08:21:13.453814 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.453775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg" event={"ID":"68464c35-0bb2-4961-aebc-36335c72a0f9","Type":"ContainerStarted","Data":"73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a"}
Sep 14 08:21:13.453984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.453843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg" event={"ID":"68464c35-0bb2-4961-aebc-36335c72a0f9","Type":"ContainerStarted","Data":"8cc715775784a949b18364c4c169c0314132f55a24145490ecaba8373e814750"}
Sep 14 08:21:13.453984 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.453873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:13.455867 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.455836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s" event={"ID":"00aca3d7-aec8-4f0d-8c13-7d00ed43678b","Type":"ContainerStarted","Data":"0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585"}
Sep 14 08:21:13.456016 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.455888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s" event={"ID":"00aca3d7-aec8-4f0d-8c13-7d00ed43678b","Type":"ContainerStarted","Data":"522fbed8db5eb67e75e17ef5498b523331533f608ae1b201126b0804740136de"}
Sep 14 08:21:13.456016 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.455990    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:13.471228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.471137    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg" podStartSLOduration=1.471120349 podStartE2EDuration="1.471120349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:21:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:21:13.468883592 +0000 UTC m=+1929.176897731" watchObservedRunningTime="2026-09-14 08:21:13.471120349 +0000 UTC m=+1929.179134488"
Sep 14 08:21:13.485290 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:13.485245    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s" podStartSLOduration=1.485230672 podStartE2EDuration="1.485230672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:21:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:21:13.483647689 +0000 UTC m=+1929.191661829" watchObservedRunningTime="2026-09-14 08:21:13.485230672 +0000 UTC m=+1929.193244811"
Sep 14 08:21:24.463889 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:24.463858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:24.464460 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:24.463935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:34.685798 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.685760    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"]
Sep 14 08:21:34.689873 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.689849    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:34.692704 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.692675    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:21:34.693381 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.693354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:21:34.695598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.695572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"]
Sep 14 08:21:34.754925 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.754900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c88e8d9-8d43-439d-a3e9-260963217ce2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf\" (UID: \"9c88e8d9-8d43-439d-a3e9-260963217ce2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:34.856245 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.856205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c88e8d9-8d43-439d-a3e9-260963217ce2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf\" (UID: \"9c88e8d9-8d43-439d-a3e9-260963217ce2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:34.856906 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:34.856884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c88e8d9-8d43-439d-a3e9-260963217ce2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf\" (UID: \"9c88e8d9-8d43-439d-a3e9-260963217ce2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:35.003505 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:35.003418    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:35.274115 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:35.274088    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"]
Sep 14 08:21:35.275716 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:21:35.275683    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c88e8d9_8d43_439d_a3e9_260963217ce2.slice/crio-1534b1985f0feef1b12fdf6e037cf967745bed92859eb4563ae4903770cf9dc0 WatchSource:0}: Error finding container 1534b1985f0feef1b12fdf6e037cf967745bed92859eb4563ae4903770cf9dc0: Status 404 returned error can't find the container with id 1534b1985f0feef1b12fdf6e037cf967745bed92859eb4563ae4903770cf9dc0
Sep 14 08:21:35.578622 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:35.578543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf" event={"ID":"9c88e8d9-8d43-439d-a3e9-260963217ce2","Type":"ContainerStarted","Data":"35b99d4fcca08d31299995c1b55d766176cb7a10ab8da45ef8830253d0bdfc1e"}
Sep 14 08:21:35.578622 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:35.578595    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf" event={"ID":"9c88e8d9-8d43-439d-a3e9-260963217ce2","Type":"ContainerStarted","Data":"1534b1985f0feef1b12fdf6e037cf967745bed92859eb4563ae4903770cf9dc0"}
Sep 14 08:21:35.595152 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:35.595105    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf" podStartSLOduration=1.595088911 podStartE2EDuration="1.595088911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:21:35.594530455 +0000 UTC m=+1951.302544600" watchObservedRunningTime="2026-09-14 08:21:35.595088911 +0000 UTC m=+1951.303103050"
Sep 14 08:21:36.585134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:36.585091    2563 generic.go:356] "Generic (PLEG): container finished" podID="9c88e8d9-8d43-439d-a3e9-260963217ce2" containerID="35b99d4fcca08d31299995c1b55d766176cb7a10ab8da45ef8830253d0bdfc1e" exitCode=0
Sep 14 08:21:36.585503 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:36.585141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf" event={"ID":"9c88e8d9-8d43-439d-a3e9-260963217ce2","Type":"ContainerDied","Data":"35b99d4fcca08d31299995c1b55d766176cb7a10ab8da45ef8830253d0bdfc1e"}
Sep 14 08:21:37.725718 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:37.725695    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:37.783008 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:37.782978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c88e8d9-8d43-439d-a3e9-260963217ce2-maas-api-service-ca\") pod \"9c88e8d9-8d43-439d-a3e9-260963217ce2\" (UID: \"9c88e8d9-8d43-439d-a3e9-260963217ce2\") "
Sep 14 08:21:37.783292 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:37.783273    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c88e8d9-8d43-439d-a3e9-260963217ce2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9c88e8d9-8d43-439d-a3e9-260963217ce2" (UID: "9c88e8d9-8d43-439d-a3e9-260963217ce2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:37.884777 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:37.884693    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c88e8d9-8d43-439d-a3e9-260963217ce2-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:21:38.597266 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:38.597237    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"
Sep 14 08:21:38.597266 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:38.597248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf" event={"ID":"9c88e8d9-8d43-439d-a3e9-260963217ce2","Type":"ContainerDied","Data":"1534b1985f0feef1b12fdf6e037cf967745bed92859eb4563ae4903770cf9dc0"}
Sep 14 08:21:38.597472 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:38.597289    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1534b1985f0feef1b12fdf6e037cf967745bed92859eb4563ae4903770cf9dc0"
Sep 14 08:21:40.922254 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:40.922215    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"]
Sep 14 08:21:40.922750 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:40.922512    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s" podUID="00aca3d7-aec8-4f0d-8c13-7d00ed43678b" containerName="payload-processing" containerID="cri-o://0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585" gracePeriod=30
Sep 14 08:21:40.928832 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:40.928801    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"]
Sep 14 08:21:40.929037 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:40.929013    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg" podUID="68464c35-0bb2-4961-aebc-36335c72a0f9" containerName="payload-pre-processing" containerID="cri-o://73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a" gracePeriod=30
Sep 14 08:21:41.068897 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.068865    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"]
Sep 14 08:21:41.072181 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.072139    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-70893f-55hjf"]
Sep 14 08:21:41.209114 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.209090    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:41.212829 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.212808    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:41.315057 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.315027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsd98\" (UniqueName: \"kubernetes.io/projected/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-kube-api-access-fsd98\") pod \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") "
Sep 14 08:21:41.315226 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.315102    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f642n\" (UniqueName: \"kubernetes.io/projected/68464c35-0bb2-4961-aebc-36335c72a0f9-kube-api-access-f642n\") pod \"68464c35-0bb2-4961-aebc-36335c72a0f9\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") "
Sep 14 08:21:41.315226 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.315182    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-plugins-config-volume\") pod \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\" (UID: \"00aca3d7-aec8-4f0d-8c13-7d00ed43678b\") "
Sep 14 08:21:41.315226 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.315200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68464c35-0bb2-4961-aebc-36335c72a0f9-plugins-config-volume\") pod \"68464c35-0bb2-4961-aebc-36335c72a0f9\" (UID: \"68464c35-0bb2-4961-aebc-36335c72a0f9\") "
Sep 14 08:21:41.315592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.315561    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00aca3d7-aec8-4f0d-8c13-7d00ed43678b" (UID: "00aca3d7-aec8-4f0d-8c13-7d00ed43678b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:41.315592 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.315582    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68464c35-0bb2-4961-aebc-36335c72a0f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68464c35-0bb2-4961-aebc-36335c72a0f9" (UID: "68464c35-0bb2-4961-aebc-36335c72a0f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:41.317210 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.317184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-kube-api-access-fsd98" (OuterVolumeSpecName: "kube-api-access-fsd98") pod "00aca3d7-aec8-4f0d-8c13-7d00ed43678b" (UID: "00aca3d7-aec8-4f0d-8c13-7d00ed43678b"). InnerVolumeSpecName "kube-api-access-fsd98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:41.317303 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.317225    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68464c35-0bb2-4961-aebc-36335c72a0f9-kube-api-access-f642n" (OuterVolumeSpecName: "kube-api-access-f642n") pod "68464c35-0bb2-4961-aebc-36335c72a0f9" (UID: "68464c35-0bb2-4961-aebc-36335c72a0f9"). InnerVolumeSpecName "kube-api-access-f642n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:41.416270 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.416243    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.416270 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.416267    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68464c35-0bb2-4961-aebc-36335c72a0f9-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.416424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.416277    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsd98\" (UniqueName: \"kubernetes.io/projected/00aca3d7-aec8-4f0d-8c13-7d00ed43678b-kube-api-access-fsd98\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.416424 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.416286    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f642n\" (UniqueName: \"kubernetes.io/projected/68464c35-0bb2-4961-aebc-36335c72a0f9-kube-api-access-f642n\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:21:41.615826 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.615714    2563 generic.go:356] "Generic (PLEG): container finished" podID="68464c35-0bb2-4961-aebc-36335c72a0f9" containerID="73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a" exitCode=0
Sep 14 08:21:41.615826 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.615779    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"
Sep 14 08:21:41.615826 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.615795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg" event={"ID":"68464c35-0bb2-4961-aebc-36335c72a0f9","Type":"ContainerDied","Data":"73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a"}
Sep 14 08:21:41.616092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.615849    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg" event={"ID":"68464c35-0bb2-4961-aebc-36335c72a0f9","Type":"ContainerDied","Data":"8cc715775784a949b18364c4c169c0314132f55a24145490ecaba8373e814750"}
Sep 14 08:21:41.616092 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.615871    2563 scope.go:117] "RemoveContainer" containerID="73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a"
Sep 14 08:21:41.617391 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.617366    2563 generic.go:356] "Generic (PLEG): container finished" podID="00aca3d7-aec8-4f0d-8c13-7d00ed43678b" containerID="0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585" exitCode=0
Sep 14 08:21:41.617508 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.617402    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"
Sep 14 08:21:41.617508 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.617402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s" event={"ID":"00aca3d7-aec8-4f0d-8c13-7d00ed43678b","Type":"ContainerDied","Data":"0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585"}
Sep 14 08:21:41.617583 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.617537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s" event={"ID":"00aca3d7-aec8-4f0d-8c13-7d00ed43678b","Type":"ContainerDied","Data":"522fbed8db5eb67e75e17ef5498b523331533f608ae1b201126b0804740136de"}
Sep 14 08:21:41.629491 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.629469    2563 scope.go:117] "RemoveContainer" containerID="73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a"
Sep 14 08:21:41.629896 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:21:41.629868    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a\": container with ID starting with 73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a not found: ID does not exist" containerID="73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a"
Sep 14 08:21:41.629996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.629907    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a"} err="failed to get container status \"73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a\": rpc error: code = NotFound desc = could not find container \"73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a\": container with ID starting with 73e59a4771b184da0e1dbea0a95f34ca33d549628105353183bde9cf18a1f80a not found: ID does not exist"
Sep 14 08:21:41.629996 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.629930    2563 scope.go:117] "RemoveContainer" containerID="0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585"
Sep 14 08:21:41.638453 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.638402    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"]
Sep 14 08:21:41.638508 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.638492    2563 scope.go:117] "RemoveContainer" containerID="0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585"
Sep 14 08:21:41.638806 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:21:41.638760    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585\": container with ID starting with 0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585 not found: ID does not exist" containerID="0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585"
Sep 14 08:21:41.638889 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.638817    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585"} err="failed to get container status \"0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585\": rpc error: code = NotFound desc = could not find container \"0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585\": container with ID starting with 0af380ddd240ada2268dbbb81a9806967d5e6bf696a6850548aba9329c120585 not found: ID does not exist"
Sep 14 08:21:41.641799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.641778    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-70893f-67cb9f4658-z8hxg"]
Sep 14 08:21:41.650962 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.650942    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"]
Sep 14 08:21:41.654504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:41.654485    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-70893f-678f67f845-cwc2s"]
Sep 14 08:21:42.789041 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:42.789011    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00aca3d7-aec8-4f0d-8c13-7d00ed43678b" path="/var/lib/kubelet/pods/00aca3d7-aec8-4f0d-8c13-7d00ed43678b/volumes"
Sep 14 08:21:42.789410 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:42.789382    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68464c35-0bb2-4961-aebc-36335c72a0f9" path="/var/lib/kubelet/pods/68464c35-0bb2-4961-aebc-36335c72a0f9/volumes"
Sep 14 08:21:42.789673 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:42.789660    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c88e8d9-8d43-439d-a3e9-260963217ce2" path="/var/lib/kubelet/pods/9c88e8d9-8d43-439d-a3e9-260963217ce2/volumes"
Sep 14 08:21:51.366003 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.365970    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"]
Sep 14 08:21:51.366413 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366399    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c88e8d9-8d43-439d-a3e9-260963217ce2" containerName="revoke-keys"
Sep 14 08:21:51.366413 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366414    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c88e8d9-8d43-439d-a3e9-260963217ce2" containerName="revoke-keys"
Sep 14 08:21:51.366490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366432    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68464c35-0bb2-4961-aebc-36335c72a0f9" containerName="payload-pre-processing"
Sep 14 08:21:51.366490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366438    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="68464c35-0bb2-4961-aebc-36335c72a0f9" containerName="payload-pre-processing"
Sep 14 08:21:51.366490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366447    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00aca3d7-aec8-4f0d-8c13-7d00ed43678b" containerName="payload-processing"
Sep 14 08:21:51.366490 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366454    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="00aca3d7-aec8-4f0d-8c13-7d00ed43678b" containerName="payload-processing"
Sep 14 08:21:51.366614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366514    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="68464c35-0bb2-4961-aebc-36335c72a0f9" containerName="payload-pre-processing"
Sep 14 08:21:51.366614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366529    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="00aca3d7-aec8-4f0d-8c13-7d00ed43678b" containerName="payload-processing"
Sep 14 08:21:51.366614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.366536    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c88e8d9-8d43-439d-a3e9-260963217ce2" containerName="revoke-keys"
Sep 14 08:21:51.371297 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.371280    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:51.374390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.374090    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:21:51.374390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.374134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:21:51.375716 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.375687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"]
Sep 14 08:21:51.510407 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.510377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6\" (UID: \"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:51.611888 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.611856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6\" (UID: \"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:51.612461 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.612441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6\" (UID: \"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:51.683541 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.683518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:51.956742 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:51.956694    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"]
Sep 14 08:21:51.958211 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:21:51.958180    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c0969f_472a_4dcc_ba94_232fbbdcc9f5.slice/crio-19bf8d853570dda48b291de61693be683519de69e3721eb44ea8c331d9bd3fca WatchSource:0}: Error finding container 19bf8d853570dda48b291de61693be683519de69e3721eb44ea8c331d9bd3fca: Status 404 returned error can't find the container with id 19bf8d853570dda48b291de61693be683519de69e3721eb44ea8c331d9bd3fca
Sep 14 08:21:52.683020 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:52.682988    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8c0969f-472a-4dcc-ba94-232fbbdcc9f5" containerID="b4edb8d79e7074fb8819d3de293ee57a08ddfeedf5f7d71260b43bf2b9cb7c6c" exitCode=0
Sep 14 08:21:52.683330 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:52.683052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6" event={"ID":"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5","Type":"ContainerDied","Data":"b4edb8d79e7074fb8819d3de293ee57a08ddfeedf5f7d71260b43bf2b9cb7c6c"}
Sep 14 08:21:52.683330 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:52.683092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6" event={"ID":"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5","Type":"ContainerStarted","Data":"19bf8d853570dda48b291de61693be683519de69e3721eb44ea8c331d9bd3fca"}
Sep 14 08:21:53.830011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:53.829977    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:53.938556 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:53.938521    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5-maas-api-service-ca\") pod \"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5\" (UID: \"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5\") "
Sep 14 08:21:53.939009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:53.938954    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8c0969f-472a-4dcc-ba94-232fbbdcc9f5" (UID: "f8c0969f-472a-4dcc-ba94-232fbbdcc9f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:54.039468 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:54.039438    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:21:54.695400 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:54.695370    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"
Sep 14 08:21:54.695400 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:54.695386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6" event={"ID":"f8c0969f-472a-4dcc-ba94-232fbbdcc9f5","Type":"ContainerDied","Data":"19bf8d853570dda48b291de61693be683519de69e3721eb44ea8c331d9bd3fca"}
Sep 14 08:21:54.695648 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:21:54.695428    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19bf8d853570dda48b291de61693be683519de69e3721eb44ea8c331d9bd3fca"
Sep 14 08:22:05.253091 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:05.253014    2563 scope.go:117] "RemoveContainer" containerID="ad090dc5354b3e0a62f83ef1049c0ac06150d503be70313b13906190c1d36e74"
Sep 14 08:22:07.117342 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:07.117303    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"]
Sep 14 08:22:07.119978 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:07.119955    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-03d6fa-6bqf6"]
Sep 14 08:22:08.789776 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:08.789741    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8c0969f-472a-4dcc-ba94-232fbbdcc9f5" path="/var/lib/kubelet/pods/f8c0969f-472a-4dcc-ba94-232fbbdcc9f5/volumes"
Sep 14 08:22:12.489926 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:12.489893    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"]
Sep 14 08:22:27.509219 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.509172    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7" podUID="17cf0e76-60b7-417b-ae8b-2b85a0f66c07" containerName="main" containerID="cri-o://e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17" gracePeriod=45
Sep 14 08:22:27.760716 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.760650    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:22:27.855261 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-dshm\") pod \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") "
Sep 14 08:22:27.855407 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855272    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-home\") pod \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") "
Sep 14 08:22:27.855407 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855324    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wrbz\" (UniqueName: \"kubernetes.io/projected/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-kube-api-access-8wrbz\") pod \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") "
Sep 14 08:22:27.855407 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tmp-dir\") pod \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") "
Sep 14 08:22:27.855407 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855389    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tls-certs\") pod \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") "
Sep 14 08:22:27.855624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855435    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-model-cache\") pod \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\" (UID: \"17cf0e76-60b7-417b-ae8b-2b85a0f66c07\") "
Sep 14 08:22:27.855624 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855533    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-home" (OuterVolumeSpecName: "home") pod "17cf0e76-60b7-417b-ae8b-2b85a0f66c07" (UID: "17cf0e76-60b7-417b-ae8b-2b85a0f66c07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:22:27.855751 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "17cf0e76-60b7-417b-ae8b-2b85a0f66c07" (UID: "17cf0e76-60b7-417b-ae8b-2b85a0f66c07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:22:27.855751 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855704    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-model-cache" (OuterVolumeSpecName: "model-cache") pod "17cf0e76-60b7-417b-ae8b-2b85a0f66c07" (UID: "17cf0e76-60b7-417b-ae8b-2b85a0f66c07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:22:27.855915 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855881    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-home\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:22:27.855915 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855905    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tmp-dir\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:22:27.855915 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.855921    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-model-cache\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:22:27.857452 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.857429    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-dshm" (OuterVolumeSpecName: "dshm") pod "17cf0e76-60b7-417b-ae8b-2b85a0f66c07" (UID: "17cf0e76-60b7-417b-ae8b-2b85a0f66c07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:22:27.857531 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.857434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-kube-api-access-8wrbz" (OuterVolumeSpecName: "kube-api-access-8wrbz") pod "17cf0e76-60b7-417b-ae8b-2b85a0f66c07" (UID: "17cf0e76-60b7-417b-ae8b-2b85a0f66c07"). InnerVolumeSpecName "kube-api-access-8wrbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:22:27.857531 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.857453    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "17cf0e76-60b7-417b-ae8b-2b85a0f66c07" (UID: "17cf0e76-60b7-417b-ae8b-2b85a0f66c07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:22:27.882483 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.882456    2563 generic.go:356] "Generic (PLEG): container finished" podID="17cf0e76-60b7-417b-ae8b-2b85a0f66c07" containerID="e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17" exitCode=0
Sep 14 08:22:27.882563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.882502    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"
Sep 14 08:22:27.882563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.882524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7" event={"ID":"17cf0e76-60b7-417b-ae8b-2b85a0f66c07","Type":"ContainerDied","Data":"e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17"}
Sep 14 08:22:27.882563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.882559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7" event={"ID":"17cf0e76-60b7-417b-ae8b-2b85a0f66c07","Type":"ContainerDied","Data":"20abbc28f1eb31fdf329c9849b678bef6aae55a00b04f69d1c8c3f434343ac5b"}
Sep 14 08:22:27.882670 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.882582    2563 scope.go:117] "RemoveContainer" containerID="e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17"
Sep 14 08:22:27.891293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.891275    2563 scope.go:117] "RemoveContainer" containerID="e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17"
Sep 14 08:22:27.891557 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:22:27.891542    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17\": container with ID starting with e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17 not found: ID does not exist" containerID="e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17"
Sep 14 08:22:27.891598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.891567    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17"} err="failed to get container status \"e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17\": rpc error: code = NotFound desc = could not find container \"e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17\": container with ID starting with e70d7ddf6f6ce03e671364f020d0a2db3a552aceeee9f47a7cc7dbafd76b5d17 not found: ID does not exist"
Sep 14 08:22:27.902939 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.902916    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"]
Sep 14 08:22:27.906985 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.906965    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-03d6fa/ipp-route-03d6fa-kserve-56965b9cf-466s7"]
Sep 14 08:22:27.957109 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.957086    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wrbz\" (UniqueName: \"kubernetes.io/projected/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-kube-api-access-8wrbz\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:22:27.957233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.957112    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-tls-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:22:27.957233 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:27.957127    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17cf0e76-60b7-417b-ae8b-2b85a0f66c07-dshm\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:22:28.788545 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:22:28.788510    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17cf0e76-60b7-417b-ae8b-2b85a0f66c07" path="/var/lib/kubelet/pods/17cf0e76-60b7-417b-ae8b-2b85a0f66c07/volumes"
Sep 14 08:23:09.373432 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.373392    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"]
Sep 14 08:23:09.373933 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.373674    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy" containerID="cri-o://4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874" gracePeriod=30
Sep 14 08:23:09.773891 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.773857    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"]
Sep 14 08:23:09.774315 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.774300    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17cf0e76-60b7-417b-ae8b-2b85a0f66c07" containerName="main"
Sep 14 08:23:09.774362 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.774316    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17cf0e76-60b7-417b-ae8b-2b85a0f66c07" containerName="main"
Sep 14 08:23:09.774362 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.774346    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8c0969f-472a-4dcc-ba94-232fbbdcc9f5" containerName="revoke-keys"
Sep 14 08:23:09.774362 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.774352    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8c0969f-472a-4dcc-ba94-232fbbdcc9f5" containerName="revoke-keys"
Sep 14 08:23:09.774463 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.774412    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17cf0e76-60b7-417b-ae8b-2b85a0f66c07" containerName="main"
Sep 14 08:23:09.774463 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.774422    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8c0969f-472a-4dcc-ba94-232fbbdcc9f5" containerName="revoke-keys"
Sep 14 08:23:09.777636 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.777614    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:09.779811 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.779789    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:23:09.779922 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.779791    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:23:09.784962 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.784935    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"]
Sep 14 08:23:09.948940 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:09.948906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bef9fdc7-dc41-4504-a6bb-1c943aad39e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt\" (UID: \"bef9fdc7-dc41-4504-a6bb-1c943aad39e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:10.050235 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:10.050153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bef9fdc7-dc41-4504-a6bb-1c943aad39e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt\" (UID: \"bef9fdc7-dc41-4504-a6bb-1c943aad39e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:10.050788 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:10.050768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bef9fdc7-dc41-4504-a6bb-1c943aad39e7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt\" (UID: \"bef9fdc7-dc41-4504-a6bb-1c943aad39e7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:10.088792 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:10.088774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:10.358824 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:10.358800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"]
Sep 14 08:23:10.360327 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:23:10.360300    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef9fdc7_dc41_4504_a6bb_1c943aad39e7.slice/crio-e756190fa5acac1e2ea7286fb548ecb749b0e9702f2b746ed23b8c0a5435d9f3 WatchSource:0}: Error finding container e756190fa5acac1e2ea7286fb548ecb749b0e9702f2b746ed23b8c0a5435d9f3: Status 404 returned error can't find the container with id e756190fa5acac1e2ea7286fb548ecb749b0e9702f2b746ed23b8c0a5435d9f3
Sep 14 08:23:11.117938 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:11.117904    2563 generic.go:356] "Generic (PLEG): container finished" podID="bef9fdc7-dc41-4504-a6bb-1c943aad39e7" containerID="f71036a03217e081843edc70e93c22c8eea13641ad2593a750cfcc97fb4fdbb5" exitCode=0
Sep 14 08:23:11.118293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:11.117995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt" event={"ID":"bef9fdc7-dc41-4504-a6bb-1c943aad39e7","Type":"ContainerDied","Data":"f71036a03217e081843edc70e93c22c8eea13641ad2593a750cfcc97fb4fdbb5"}
Sep 14 08:23:11.118293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:11.118078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt" event={"ID":"bef9fdc7-dc41-4504-a6bb-1c943aad39e7","Type":"ContainerStarted","Data":"e756190fa5acac1e2ea7286fb548ecb749b0e9702f2b746ed23b8c0a5435d9f3"}
Sep 14 08:23:12.259623 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:12.259600    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:12.373042 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:12.373009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bef9fdc7-dc41-4504-a6bb-1c943aad39e7-maas-api-service-ca\") pod \"bef9fdc7-dc41-4504-a6bb-1c943aad39e7\" (UID: \"bef9fdc7-dc41-4504-a6bb-1c943aad39e7\") "
Sep 14 08:23:12.373391 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:12.373364    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef9fdc7-dc41-4504-a6bb-1c943aad39e7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bef9fdc7-dc41-4504-a6bb-1c943aad39e7" (UID: "bef9fdc7-dc41-4504-a6bb-1c943aad39e7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:12.474306 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:12.474264    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bef9fdc7-dc41-4504-a6bb-1c943aad39e7-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:13.129993 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:13.129916    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"
Sep 14 08:23:13.130140 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:13.129915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt" event={"ID":"bef9fdc7-dc41-4504-a6bb-1c943aad39e7","Type":"ContainerDied","Data":"e756190fa5acac1e2ea7286fb548ecb749b0e9702f2b746ed23b8c0a5435d9f3"}
Sep 14 08:23:13.130140 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:13.130051    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e756190fa5acac1e2ea7286fb548ecb749b0e9702f2b746ed23b8c0a5435d9f3"
Sep 14 08:23:14.628499 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.628474    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:23:14.794255 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21431a6-a018-444c-b613-6f7e3a952667-istio-podinfo\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794415 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794307    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-envoy\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794415 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21431a6-a018-444c-b613-6f7e3a952667-istiod-ca-cert\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794503 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-certs\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794503 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794469    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-credential-socket\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794503 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794488    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-data\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-istio-token\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794559    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-socket\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794647 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794591    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6k9h\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-kube-api-access-h6k9h\") pod \"d21431a6-a018-444c-b613-6f7e3a952667\" (UID: \"d21431a6-a018-444c-b613-6f7e3a952667\") "
Sep 14 08:23:14.794830 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794769    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-data" (OuterVolumeSpecName: "istio-data") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:14.794830 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794783    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:14.794941 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.794834    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21431a6-a018-444c-b613-6f7e3a952667-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:14.795067 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.795006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:14.795067 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.795016    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21431a6-a018-444c-b613-6f7e3a952667-istiod-ca-cert\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.795067 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.795035    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.795067 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.795044    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-data\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.795255 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.795058    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:14.796790 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.796759    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-istio-token" (OuterVolumeSpecName: "istio-token") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:14.796901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.796785    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-kube-api-access-h6k9h" (OuterVolumeSpecName: "kube-api-access-h6k9h") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "kube-api-access-h6k9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:14.796971 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.796955    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:23:14.797011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.796989    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d21431a6-a018-444c-b613-6f7e3a952667-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d21431a6-a018-444c-b613-6f7e3a952667" (UID: "d21431a6-a018-444c-b613-6f7e3a952667"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:23:14.896614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.896581    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-istio-token\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.896614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.896611    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-workload-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.896881 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.896702    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6k9h\" (UniqueName: \"kubernetes.io/projected/d21431a6-a018-444c-b613-6f7e3a952667-kube-api-access-h6k9h\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.896881 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.896720    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21431a6-a018-444c-b613-6f7e3a952667-istio-podinfo\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.896881 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.896762    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-istio-envoy\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:14.896881 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:14.896774    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21431a6-a018-444c-b613-6f7e3a952667-credential-socket\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:15.142128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.142043    2563 generic.go:356] "Generic (PLEG): container finished" podID="d21431a6-a018-444c-b613-6f7e3a952667" containerID="4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874" exitCode=0
Sep 14 08:23:15.142128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.142099    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"
Sep 14 08:23:15.142327 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.142123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" event={"ID":"d21431a6-a018-444c-b613-6f7e3a952667","Type":"ContainerDied","Data":"4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874"}
Sep 14 08:23:15.142327 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.142174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm" event={"ID":"d21431a6-a018-444c-b613-6f7e3a952667","Type":"ContainerDied","Data":"cc79b05ae7c231812d2bb819f510f6f5772e0dfca7c3567e55b53e4ca4cd1a96"}
Sep 14 08:23:15.142327 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.142198    2563 scope.go:117] "RemoveContainer" containerID="4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874"
Sep 14 08:23:15.152363 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.152348    2563 scope.go:117] "RemoveContainer" containerID="6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"
Sep 14 08:23:15.161454 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.161422    2563 scope.go:117] "RemoveContainer" containerID="4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874"
Sep 14 08:23:15.161777 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:23:15.161751    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874\": container with ID starting with 4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874 not found: ID does not exist" containerID="4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874"
Sep 14 08:23:15.161853 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.161807    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874"} err="failed to get container status \"4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874\": rpc error: code = NotFound desc = could not find container \"4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874\": container with ID starting with 4d8f034c87f6087e3a8b074e631dd4f1b288a3f9c0dab0114138e65e9e7e6874 not found: ID does not exist"
Sep 14 08:23:15.161853 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.161824    2563 scope.go:117] "RemoveContainer" containerID="6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"
Sep 14 08:23:15.162172 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:23:15.162139    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706\": container with ID starting with 6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706 not found: ID does not exist" containerID="6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"
Sep 14 08:23:15.162224 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.162169    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706"} err="failed to get container status \"6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706\": rpc error: code = NotFound desc = could not find container \"6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706\": container with ID starting with 6857920e7ca8b17de31d59f0dcce22ad0fd464045c6cfbc0dd6a6c64c4cda706 not found: ID does not exist"
Sep 14 08:23:15.163681 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.163664    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"]
Sep 14 08:23:15.169954 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:15.169928    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-03d6fa-openshift-default-d8b6f7fb6-bltcm"]
Sep 14 08:23:16.789682 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:16.789652    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21431a6-a018-444c-b613-6f7e3a952667" path="/var/lib/kubelet/pods/d21431a6-a018-444c-b613-6f7e3a952667/volumes"
Sep 14 08:23:20.283201 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.283166    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"]
Sep 14 08:23:20.283644 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.283359    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp" podUID="74fb50b0-6409-4ffa-a3d4-05daf19492e9" containerName="payload-pre-processing" containerID="cri-o://533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f" gracePeriod=30
Sep 14 08:23:20.518328 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.518295    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"]
Sep 14 08:23:20.520128 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.520105    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-7bcf7b-mc2gt"]
Sep 14 08:23:20.538268 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.538183    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:23:20.546747 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.546710    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8whx\" (UniqueName: \"kubernetes.io/projected/74fb50b0-6409-4ffa-a3d4-05daf19492e9-kube-api-access-s8whx\") pod \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") "
Sep 14 08:23:20.546892 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.546804    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74fb50b0-6409-4ffa-a3d4-05daf19492e9-plugins-config-volume\") pod \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\" (UID: \"74fb50b0-6409-4ffa-a3d4-05daf19492e9\") "
Sep 14 08:23:20.547161 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.547141    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74fb50b0-6409-4ffa-a3d4-05daf19492e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74fb50b0-6409-4ffa-a3d4-05daf19492e9" (UID: "74fb50b0-6409-4ffa-a3d4-05daf19492e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:20.548950 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.548927    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74fb50b0-6409-4ffa-a3d4-05daf19492e9-kube-api-access-s8whx" (OuterVolumeSpecName: "kube-api-access-s8whx") pod "74fb50b0-6409-4ffa-a3d4-05daf19492e9" (UID: "74fb50b0-6409-4ffa-a3d4-05daf19492e9"). InnerVolumeSpecName "kube-api-access-s8whx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:20.647849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.647816    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74fb50b0-6409-4ffa-a3d4-05daf19492e9-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.647849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.647842    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8whx\" (UniqueName: \"kubernetes.io/projected/74fb50b0-6409-4ffa-a3d4-05daf19492e9-kube-api-access-s8whx\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:20.788614 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:20.788537    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bef9fdc7-dc41-4504-a6bb-1c943aad39e7" path="/var/lib/kubelet/pods/bef9fdc7-dc41-4504-a6bb-1c943aad39e7/volumes"
Sep 14 08:23:21.176711 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.176622    2563 generic.go:356] "Generic (PLEG): container finished" podID="74fb50b0-6409-4ffa-a3d4-05daf19492e9" containerID="533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f" exitCode=0
Sep 14 08:23:21.176711 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.176664    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"
Sep 14 08:23:21.176711 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.176700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp" event={"ID":"74fb50b0-6409-4ffa-a3d4-05daf19492e9","Type":"ContainerDied","Data":"533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f"}
Sep 14 08:23:21.176971 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.176775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp" event={"ID":"74fb50b0-6409-4ffa-a3d4-05daf19492e9","Type":"ContainerDied","Data":"2e13641e3a3af795881d9c9f889f9dc2344892e065904fff986f9df644691a63"}
Sep 14 08:23:21.176971 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.176801    2563 scope.go:117] "RemoveContainer" containerID="533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f"
Sep 14 08:23:21.185717 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.185701    2563 scope.go:117] "RemoveContainer" containerID="533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f"
Sep 14 08:23:21.185973 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:23:21.185957    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f\": container with ID starting with 533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f not found: ID does not exist" containerID="533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f"
Sep 14 08:23:21.186028 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.185980    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f"} err="failed to get container status \"533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f\": rpc error: code = NotFound desc = could not find container \"533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f\": container with ID starting with 533ce6eb731324829093a8a83965fa3f1148591aaf6aa140c38e902ef7cf562f not found: ID does not exist"
Sep 14 08:23:21.192442 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.192420    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"]
Sep 14 08:23:21.197080 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:21.197061    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-7bcf7b-54dd6fcc98-st8fp"]
Sep 14 08:23:22.789888 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:22.789857    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74fb50b0-6409-4ffa-a3d4-05daf19492e9" path="/var/lib/kubelet/pods/74fb50b0-6409-4ffa-a3d4-05daf19492e9/volumes"
Sep 14 08:23:27.160603 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160522    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"]
Sep 14 08:23:27.160976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160939    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bef9fdc7-dc41-4504-a6bb-1c943aad39e7" containerName="revoke-keys"
Sep 14 08:23:27.160976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160951    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef9fdc7-dc41-4504-a6bb-1c943aad39e7" containerName="revoke-keys"
Sep 14 08:23:27.160976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160963    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy"
Sep 14 08:23:27.160976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160969    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy"
Sep 14 08:23:27.160976 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160976    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74fb50b0-6409-4ffa-a3d4-05daf19492e9" containerName="payload-pre-processing"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160981    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="74fb50b0-6409-4ffa-a3d4-05daf19492e9" containerName="payload-pre-processing"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.160997    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.161002    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.161076    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bef9fdc7-dc41-4504-a6bb-1c943aad39e7" containerName="revoke-keys"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.161087    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.161094    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21431a6-a018-444c-b613-6f7e3a952667" containerName="istio-proxy"
Sep 14 08:23:27.161134 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.161100    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="74fb50b0-6409-4ffa-a3d4-05daf19492e9" containerName="payload-pre-processing"
Sep 14 08:23:27.165504 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.165487    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:27.169056 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.169018    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:23:27.169184 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.169134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:23:27.182897 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.182870    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"]
Sep 14 08:23:27.208405 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.208378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6c64cf-9087-4032-97eb-39d448d4ff8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw\" (UID: \"6c6c64cf-9087-4032-97eb-39d448d4ff8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:27.309556 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.309531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6c64cf-9087-4032-97eb-39d448d4ff8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw\" (UID: \"6c6c64cf-9087-4032-97eb-39d448d4ff8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:27.310155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.310137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6c64cf-9087-4032-97eb-39d448d4ff8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw\" (UID: \"6c6c64cf-9087-4032-97eb-39d448d4ff8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:27.475030 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.475004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:27.744831 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:27.744803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"]
Sep 14 08:23:27.746656 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:23:27.746623    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6c64cf_9087_4032_97eb_39d448d4ff8d.slice/crio-76a7b8b4a7d86bf28465efc5412984bc38ec56f805c4df62a6e5c6ffe6c8ad50 WatchSource:0}: Error finding container 76a7b8b4a7d86bf28465efc5412984bc38ec56f805c4df62a6e5c6ffe6c8ad50: Status 404 returned error can't find the container with id 76a7b8b4a7d86bf28465efc5412984bc38ec56f805c4df62a6e5c6ffe6c8ad50
Sep 14 08:23:28.215243 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:28.215211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw" event={"ID":"6c6c64cf-9087-4032-97eb-39d448d4ff8d","Type":"ContainerStarted","Data":"e6fcca6618f60a98b4bc53af2dc470cbd61a3a85d82e648136b030c75f5fa934"}
Sep 14 08:23:28.215606 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:28.215262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw" event={"ID":"6c6c64cf-9087-4032-97eb-39d448d4ff8d","Type":"ContainerStarted","Data":"76a7b8b4a7d86bf28465efc5412984bc38ec56f805c4df62a6e5c6ffe6c8ad50"}
Sep 14 08:23:28.230049 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:28.230006    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw" podStartSLOduration=1.229989618 podStartE2EDuration="1.229989618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:23:28.228785822 +0000 UTC m=+2063.936799972" watchObservedRunningTime="2026-09-14 08:23:28.229989618 +0000 UTC m=+2063.938003757"
Sep 14 08:23:29.221977 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:29.221940    2563 generic.go:356] "Generic (PLEG): container finished" podID="6c6c64cf-9087-4032-97eb-39d448d4ff8d" containerID="e6fcca6618f60a98b4bc53af2dc470cbd61a3a85d82e648136b030c75f5fa934" exitCode=0
Sep 14 08:23:29.222348 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:29.221999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw" event={"ID":"6c6c64cf-9087-4032-97eb-39d448d4ff8d","Type":"ContainerDied","Data":"e6fcca6618f60a98b4bc53af2dc470cbd61a3a85d82e648136b030c75f5fa934"}
Sep 14 08:23:30.358511 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:30.358486    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:30.441308 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:30.441277    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6c64cf-9087-4032-97eb-39d448d4ff8d-maas-api-service-ca\") pod \"6c6c64cf-9087-4032-97eb-39d448d4ff8d\" (UID: \"6c6c64cf-9087-4032-97eb-39d448d4ff8d\") "
Sep 14 08:23:30.441601 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:30.441579    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c6c64cf-9087-4032-97eb-39d448d4ff8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c6c64cf-9087-4032-97eb-39d448d4ff8d" (UID: "6c6c64cf-9087-4032-97eb-39d448d4ff8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:30.542206 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:30.542119    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6c64cf-9087-4032-97eb-39d448d4ff8d-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:31.234017 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:31.233987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw" event={"ID":"6c6c64cf-9087-4032-97eb-39d448d4ff8d","Type":"ContainerDied","Data":"76a7b8b4a7d86bf28465efc5412984bc38ec56f805c4df62a6e5c6ffe6c8ad50"}
Sep 14 08:23:31.234183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:31.234026    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"
Sep 14 08:23:31.234183 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:31.234040    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76a7b8b4a7d86bf28465efc5412984bc38ec56f805c4df62a6e5c6ffe6c8ad50"
Sep 14 08:23:37.806796 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:37.806761    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"]
Sep 14 08:23:37.807231 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:37.807030    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv" podUID="8db62108-decf-404a-8294-d2bbad8abc63" containerName="payload-pre-processing" containerID="cri-o://b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5" gracePeriod=30
Sep 14 08:23:37.963868 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:37.963831    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"]
Sep 14 08:23:37.965700 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:37.965674    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-919cc9-c7vgw"]
Sep 14 08:23:38.060228 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.060173    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:23:38.109612 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.109586    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r99j2\" (UniqueName: \"kubernetes.io/projected/8db62108-decf-404a-8294-d2bbad8abc63-kube-api-access-r99j2\") pod \"8db62108-decf-404a-8294-d2bbad8abc63\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") "
Sep 14 08:23:38.109743 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.109622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db62108-decf-404a-8294-d2bbad8abc63-plugins-config-volume\") pod \"8db62108-decf-404a-8294-d2bbad8abc63\" (UID: \"8db62108-decf-404a-8294-d2bbad8abc63\") "
Sep 14 08:23:38.110076 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.110055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db62108-decf-404a-8294-d2bbad8abc63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8db62108-decf-404a-8294-d2bbad8abc63" (UID: "8db62108-decf-404a-8294-d2bbad8abc63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:38.111644 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.111624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db62108-decf-404a-8294-d2bbad8abc63-kube-api-access-r99j2" (OuterVolumeSpecName: "kube-api-access-r99j2") pod "8db62108-decf-404a-8294-d2bbad8abc63" (UID: "8db62108-decf-404a-8294-d2bbad8abc63"). InnerVolumeSpecName "kube-api-access-r99j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:38.211351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.211324    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r99j2\" (UniqueName: \"kubernetes.io/projected/8db62108-decf-404a-8294-d2bbad8abc63-kube-api-access-r99j2\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.211351 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.211348    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db62108-decf-404a-8294-d2bbad8abc63-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:38.274488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.274456    2563 generic.go:356] "Generic (PLEG): container finished" podID="8db62108-decf-404a-8294-d2bbad8abc63" containerID="b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5" exitCode=0
Sep 14 08:23:38.274622 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.274502    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"
Sep 14 08:23:38.274622 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.274528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv" event={"ID":"8db62108-decf-404a-8294-d2bbad8abc63","Type":"ContainerDied","Data":"b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5"}
Sep 14 08:23:38.274622 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.274582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv" event={"ID":"8db62108-decf-404a-8294-d2bbad8abc63","Type":"ContainerDied","Data":"d848b92ea5d7302b4a564317bc84060a0c9e7de85c1d4fd1015aa5759a5080bf"}
Sep 14 08:23:38.274622 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.274611    2563 scope.go:117] "RemoveContainer" containerID="b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5"
Sep 14 08:23:38.283538 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.283521    2563 scope.go:117] "RemoveContainer" containerID="b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5"
Sep 14 08:23:38.283841 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:23:38.283821    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5\": container with ID starting with b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5 not found: ID does not exist" containerID="b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5"
Sep 14 08:23:38.283902 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.283848    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5"} err="failed to get container status \"b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5\": rpc error: code = NotFound desc = could not find container \"b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5\": container with ID starting with b2278f45309fd17639f2f0c9f414eb3952bb213c0a05a1da1f98ed267bc144f5 not found: ID does not exist"
Sep 14 08:23:38.295538 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.295518    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"]
Sep 14 08:23:38.300222 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.300202    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-919cc9-b649ffc66-pkjpv"]
Sep 14 08:23:38.789955 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.789922    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6c64cf-9087-4032-97eb-39d448d4ff8d" path="/var/lib/kubelet/pods/6c6c64cf-9087-4032-97eb-39d448d4ff8d/volumes"
Sep 14 08:23:38.790300 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:38.790279    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db62108-decf-404a-8294-d2bbad8abc63" path="/var/lib/kubelet/pods/8db62108-decf-404a-8294-d2bbad8abc63/volumes"
Sep 14 08:23:44.899403 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899367    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"]
Sep 14 08:23:44.899905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899798    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6c64cf-9087-4032-97eb-39d448d4ff8d" containerName="revoke-keys"
Sep 14 08:23:44.899905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899814    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6c64cf-9087-4032-97eb-39d448d4ff8d" containerName="revoke-keys"
Sep 14 08:23:44.899905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899832    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db62108-decf-404a-8294-d2bbad8abc63" containerName="payload-pre-processing"
Sep 14 08:23:44.899905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899838    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db62108-decf-404a-8294-d2bbad8abc63" containerName="payload-pre-processing"
Sep 14 08:23:44.899905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899904    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db62108-decf-404a-8294-d2bbad8abc63" containerName="payload-pre-processing"
Sep 14 08:23:44.900081 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.899918    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c6c64cf-9087-4032-97eb-39d448d4ff8d" containerName="revoke-keys"
Sep 14 08:23:44.903246 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.903230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:44.905464 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.905437    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rhm8z\""
Sep 14 08:23:44.905591 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.905437    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:23:44.909194 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.909170    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"]
Sep 14 08:23:44.978972 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:44.978943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf43fb-62db-40ca-92e8-e0504f992a57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q\" (UID: \"e4cf43fb-62db-40ca-92e8-e0504f992a57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:45.079999 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:45.079968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf43fb-62db-40ca-92e8-e0504f992a57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q\" (UID: \"e4cf43fb-62db-40ca-92e8-e0504f992a57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:45.080583 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:45.080562    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf43fb-62db-40ca-92e8-e0504f992a57-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q\" (UID: \"e4cf43fb-62db-40ca-92e8-e0504f992a57\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:45.216672 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:45.216644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:45.487859 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:45.487782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"]
Sep 14 08:23:45.491474 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:23:45.491450    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cf43fb_62db_40ca_92e8_e0504f992a57.slice/crio-c98f1e6022713d1ce066bf12cfdbb382a5b54f6e047aeb1c732bb0e34986161e WatchSource:0}: Error finding container c98f1e6022713d1ce066bf12cfdbb382a5b54f6e047aeb1c732bb0e34986161e: Status 404 returned error can't find the container with id c98f1e6022713d1ce066bf12cfdbb382a5b54f6e047aeb1c732bb0e34986161e
Sep 14 08:23:46.321468 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:46.321436    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4cf43fb-62db-40ca-92e8-e0504f992a57" containerID="1a1a505607521c1a6d607e353a0085663eaa1b5b78000d637e12dba2c4b51118" exitCode=0
Sep 14 08:23:46.321849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:46.321516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q" event={"ID":"e4cf43fb-62db-40ca-92e8-e0504f992a57","Type":"ContainerDied","Data":"1a1a505607521c1a6d607e353a0085663eaa1b5b78000d637e12dba2c4b51118"}
Sep 14 08:23:46.321849 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:46.321580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q" event={"ID":"e4cf43fb-62db-40ca-92e8-e0504f992a57","Type":"ContainerStarted","Data":"c98f1e6022713d1ce066bf12cfdbb382a5b54f6e047aeb1c732bb0e34986161e"}
Sep 14 08:23:47.453413 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:47.453386    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:47.603619 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:47.603534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf43fb-62db-40ca-92e8-e0504f992a57-maas-api-service-ca\") pod \"e4cf43fb-62db-40ca-92e8-e0504f992a57\" (UID: \"e4cf43fb-62db-40ca-92e8-e0504f992a57\") "
Sep 14 08:23:47.603903 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:47.603881    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4cf43fb-62db-40ca-92e8-e0504f992a57-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4cf43fb-62db-40ca-92e8-e0504f992a57" (UID: "e4cf43fb-62db-40ca-92e8-e0504f992a57"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:47.705011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:47.704980    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4cf43fb-62db-40ca-92e8-e0504f992a57-maas-api-service-ca\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:48.333428 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:48.333389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q" event={"ID":"e4cf43fb-62db-40ca-92e8-e0504f992a57","Type":"ContainerDied","Data":"c98f1e6022713d1ce066bf12cfdbb382a5b54f6e047aeb1c732bb0e34986161e"}
Sep 14 08:23:48.333428 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:48.333411    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"
Sep 14 08:23:48.333648 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:48.333475    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c98f1e6022713d1ce066bf12cfdbb382a5b54f6e047aeb1c732bb0e34986161e"
Sep 14 08:23:55.443148 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.443113    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"]
Sep 14 08:23:55.443598 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.443368    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t" podUID="87e1a029-bcad-4e62-8f4a-1524f5bb97a0" containerName="payload-processing" containerID="cri-o://582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21" gracePeriod=30
Sep 14 08:23:55.452857 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.452826    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"]
Sep 14 08:23:55.453101 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.453052    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh" podUID="20ff6c70-da36-4825-ba6c-3dfaa9505095" containerName="payload-pre-processing" containerID="cri-o://2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94" gracePeriod=30
Sep 14 08:23:55.604942 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.604887    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"]
Sep 14 08:23:55.607527 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.607497    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-fc0d6e-vjq6q"]
Sep 14 08:23:55.721652 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.721621    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:23:55.723820 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.723804    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:23:55.886843 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.886808    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-kube-api-access-4lt5n\") pod \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") "
Sep 14 08:23:55.887004 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.886871    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqvw4\" (UniqueName: \"kubernetes.io/projected/20ff6c70-da36-4825-ba6c-3dfaa9505095-kube-api-access-jqvw4\") pod \"20ff6c70-da36-4825-ba6c-3dfaa9505095\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") "
Sep 14 08:23:55.887004 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.886920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-plugins-config-volume\") pod \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\" (UID: \"87e1a029-bcad-4e62-8f4a-1524f5bb97a0\") "
Sep 14 08:23:55.887004 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.886974    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ff6c70-da36-4825-ba6c-3dfaa9505095-plugins-config-volume\") pod \"20ff6c70-da36-4825-ba6c-3dfaa9505095\" (UID: \"20ff6c70-da36-4825-ba6c-3dfaa9505095\") "
Sep 14 08:23:55.887327 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.887303    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87e1a029-bcad-4e62-8f4a-1524f5bb97a0" (UID: "87e1a029-bcad-4e62-8f4a-1524f5bb97a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:55.887409 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.887335    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20ff6c70-da36-4825-ba6c-3dfaa9505095-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20ff6c70-da36-4825-ba6c-3dfaa9505095" (UID: "20ff6c70-da36-4825-ba6c-3dfaa9505095"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:23:55.888947 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.888928    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ff6c70-da36-4825-ba6c-3dfaa9505095-kube-api-access-jqvw4" (OuterVolumeSpecName: "kube-api-access-jqvw4") pod "20ff6c70-da36-4825-ba6c-3dfaa9505095" (UID: "20ff6c70-da36-4825-ba6c-3dfaa9505095"). InnerVolumeSpecName "kube-api-access-jqvw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:55.889035 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.888976    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-kube-api-access-4lt5n" (OuterVolumeSpecName: "kube-api-access-4lt5n") pod "87e1a029-bcad-4e62-8f4a-1524f5bb97a0" (UID: "87e1a029-bcad-4e62-8f4a-1524f5bb97a0"). InnerVolumeSpecName "kube-api-access-4lt5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:23:55.988235 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.988170    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20ff6c70-da36-4825-ba6c-3dfaa9505095-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.988235 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.988198    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lt5n\" (UniqueName: \"kubernetes.io/projected/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-kube-api-access-4lt5n\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.988235 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.988209    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqvw4\" (UniqueName: \"kubernetes.io/projected/20ff6c70-da36-4825-ba6c-3dfaa9505095-kube-api-access-jqvw4\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:55.988235 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:55.988221    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e1a029-bcad-4e62-8f4a-1524f5bb97a0-plugins-config-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:23:56.379580 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.379488    2563 generic.go:356] "Generic (PLEG): container finished" podID="20ff6c70-da36-4825-ba6c-3dfaa9505095" containerID="2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94" exitCode=0
Sep 14 08:23:56.379580 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.379531    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"
Sep 14 08:23:56.379851 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.379575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh" event={"ID":"20ff6c70-da36-4825-ba6c-3dfaa9505095","Type":"ContainerDied","Data":"2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94"}
Sep 14 08:23:56.379851 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.379632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh" event={"ID":"20ff6c70-da36-4825-ba6c-3dfaa9505095","Type":"ContainerDied","Data":"a41e2dba8b516bc54de193dab00faec1b76fc21c001121200572f7563f0232be"}
Sep 14 08:23:56.379851 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.379655    2563 scope.go:117] "RemoveContainer" containerID="2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94"
Sep 14 08:23:56.381038 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.381016    2563 generic.go:356] "Generic (PLEG): container finished" podID="87e1a029-bcad-4e62-8f4a-1524f5bb97a0" containerID="582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21" exitCode=0
Sep 14 08:23:56.381155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.381060    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"
Sep 14 08:23:56.381204 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.381061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t" event={"ID":"87e1a029-bcad-4e62-8f4a-1524f5bb97a0","Type":"ContainerDied","Data":"582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21"}
Sep 14 08:23:56.381250 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.381209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t" event={"ID":"87e1a029-bcad-4e62-8f4a-1524f5bb97a0","Type":"ContainerDied","Data":"b4fc9ba051dbad1d697b8a305c73ff9eca1e002efe2f9c8d2942e7a526253111"}
Sep 14 08:23:56.390339 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.390324    2563 scope.go:117] "RemoveContainer" containerID="2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94"
Sep 14 08:23:56.390585 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:23:56.390564    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94\": container with ID starting with 2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94 not found: ID does not exist" containerID="2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94"
Sep 14 08:23:56.390689 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.390592    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94"} err="failed to get container status \"2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94\": rpc error: code = NotFound desc = could not find container \"2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94\": container with ID starting with 2743ed7176207ebd665cf8be67f7097f157269f49f06c853349d88edce9ebd94 not found: ID does not exist"
Sep 14 08:23:56.390689 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.390608    2563 scope.go:117] "RemoveContainer" containerID="582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21"
Sep 14 08:23:56.399423 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.399257    2563 scope.go:117] "RemoveContainer" containerID="582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21"
Sep 14 08:23:56.399626 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:23:56.399575    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21\": container with ID starting with 582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21 not found: ID does not exist" containerID="582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21"
Sep 14 08:23:56.399626 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.399613    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21"} err="failed to get container status \"582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21\": rpc error: code = NotFound desc = could not find container \"582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21\": container with ID starting with 582a17b415bc744caa9e0f1f71d439a5954dc4c70ac4ae7ac0f3ff4eccbe6a21 not found: ID does not exist"
Sep 14 08:23:56.401966 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.401939    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"]
Sep 14 08:23:56.403873 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.403853    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fc0d6e-79d97d7d74-kzlvh"]
Sep 14 08:23:56.413865 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.413817    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"]
Sep 14 08:23:56.416011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.415990    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-fc0d6e-7978bc86b9-jb65t"]
Sep 14 08:23:56.789777 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.789716    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ff6c70-da36-4825-ba6c-3dfaa9505095" path="/var/lib/kubelet/pods/20ff6c70-da36-4825-ba6c-3dfaa9505095/volumes"
Sep 14 08:23:56.790202 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.790183    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87e1a029-bcad-4e62-8f4a-1524f5bb97a0" path="/var/lib/kubelet/pods/87e1a029-bcad-4e62-8f4a-1524f5bb97a0/volumes"
Sep 14 08:23:56.790563 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:23:56.790550    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4cf43fb-62db-40ca-92e8-e0504f992a57" path="/var/lib/kubelet/pods/e4cf43fb-62db-40ca-92e8-e0504f992a57/volumes"
Sep 14 08:24:04.888906 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:24:04.888872    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:24:04.893536 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:24:04.893513    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:26:34.363932 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:34.363855    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"]
Sep 14 08:26:34.364447 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:34.364037    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk" podUID="600e5533-3269-418a-b954-a47ab9026920" containerName="manager" containerID="cri-o://54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9" gracePeriod=10
Sep 14 08:26:35.816383 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:35.816358    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 08:26:35.907358 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:35.907329    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/600e5533-3269-418a-b954-a47ab9026920-extensions-socket-volume\") pod \"600e5533-3269-418a-b954-a47ab9026920\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") "
Sep 14 08:26:35.907537 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:35.907382    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r56s9\" (UniqueName: \"kubernetes.io/projected/600e5533-3269-418a-b954-a47ab9026920-kube-api-access-r56s9\") pod \"600e5533-3269-418a-b954-a47ab9026920\" (UID: \"600e5533-3269-418a-b954-a47ab9026920\") "
Sep 14 08:26:35.907683 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:35.907661    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600e5533-3269-418a-b954-a47ab9026920-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "600e5533-3269-418a-b954-a47ab9026920" (UID: "600e5533-3269-418a-b954-a47ab9026920"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:35.909311 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:35.909283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600e5533-3269-418a-b954-a47ab9026920-kube-api-access-r56s9" (OuterVolumeSpecName: "kube-api-access-r56s9") pod "600e5533-3269-418a-b954-a47ab9026920" (UID: "600e5533-3269-418a-b954-a47ab9026920"). InnerVolumeSpecName "kube-api-access-r56s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:26:36.008896 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.008872    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/600e5533-3269-418a-b954-a47ab9026920-extensions-socket-volume\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:26:36.009009 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.008915    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r56s9\" (UniqueName: \"kubernetes.io/projected/600e5533-3269-418a-b954-a47ab9026920-kube-api-access-r56s9\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:26:36.212709 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.212675    2563 generic.go:356] "Generic (PLEG): container finished" podID="600e5533-3269-418a-b954-a47ab9026920" containerID="54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9" exitCode=0
Sep 14 08:26:36.212900 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.212742    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"
Sep 14 08:26:36.212900 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.212765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk" event={"ID":"600e5533-3269-418a-b954-a47ab9026920","Type":"ContainerDied","Data":"54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9"}
Sep 14 08:26:36.212900 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.212820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk" event={"ID":"600e5533-3269-418a-b954-a47ab9026920","Type":"ContainerDied","Data":"fe88bf8351e9bf63f775867f4fd16a385f60474ce0c393d00b53654324dc885b"}
Sep 14 08:26:36.212900 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.212843    2563 scope.go:117] "RemoveContainer" containerID="54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9"
Sep 14 08:26:36.222521 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.222505    2563 scope.go:117] "RemoveContainer" containerID="54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9"
Sep 14 08:26:36.222773 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:26:36.222753    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9\": container with ID starting with 54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9 not found: ID does not exist" containerID="54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9"
Sep 14 08:26:36.222844 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.222782    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9"} err="failed to get container status \"54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9\": rpc error: code = NotFound desc = could not find container \"54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9\": container with ID starting with 54a885128c4249250bd647892d2eb3ba7161d1e8acd6aaa6ac03ba863283c2e9 not found: ID does not exist"
Sep 14 08:26:36.234160 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.234137    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"]
Sep 14 08:26:36.237719 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.237698    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ksngk"]
Sep 14 08:26:36.789040 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:26:36.789010    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600e5533-3269-418a-b954-a47ab9026920" path="/var/lib/kubelet/pods/600e5533-3269-418a-b954-a47ab9026920/volumes"
Sep 14 08:27:35.102819 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.102782    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"]
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103149    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ff6c70-da36-4825-ba6c-3dfaa9505095" containerName="payload-pre-processing"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103161    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ff6c70-da36-4825-ba6c-3dfaa9505095" containerName="payload-pre-processing"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103171    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600e5533-3269-418a-b954-a47ab9026920" containerName="manager"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103177    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="600e5533-3269-418a-b954-a47ab9026920" containerName="manager"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103184    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87e1a029-bcad-4e62-8f4a-1524f5bb97a0" containerName="payload-processing"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103190    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e1a029-bcad-4e62-8f4a-1524f5bb97a0" containerName="payload-processing"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103197    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4cf43fb-62db-40ca-92e8-e0504f992a57" containerName="revoke-keys"
Sep 14 08:27:35.103215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103202    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4cf43fb-62db-40ca-92e8-e0504f992a57" containerName="revoke-keys"
Sep 14 08:27:35.103467 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103273    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="600e5533-3269-418a-b954-a47ab9026920" containerName="manager"
Sep 14 08:27:35.103467 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103283    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4cf43fb-62db-40ca-92e8-e0504f992a57" containerName="revoke-keys"
Sep 14 08:27:35.103467 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103289    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="87e1a029-bcad-4e62-8f4a-1524f5bb97a0" containerName="payload-processing"
Sep 14 08:27:35.103467 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.103299    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ff6c70-da36-4825-ba6c-3dfaa9505095" containerName="payload-pre-processing"
Sep 14 08:27:35.106401 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.106384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.108861 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.108830    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vb5sh\""
Sep 14 08:27:35.116913 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.116890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"]
Sep 14 08:27:35.221020 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.220993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f599311f-5832-43e6-9aa3-342bf7e26d36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wq8pj\" (UID: \"f599311f-5832-43e6-9aa3-342bf7e26d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.221165 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.221042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8jcr\" (UniqueName: \"kubernetes.io/projected/f599311f-5832-43e6-9aa3-342bf7e26d36-kube-api-access-t8jcr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wq8pj\" (UID: \"f599311f-5832-43e6-9aa3-342bf7e26d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.321469 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.321434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8jcr\" (UniqueName: \"kubernetes.io/projected/f599311f-5832-43e6-9aa3-342bf7e26d36-kube-api-access-t8jcr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wq8pj\" (UID: \"f599311f-5832-43e6-9aa3-342bf7e26d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.321617 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.321532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f599311f-5832-43e6-9aa3-342bf7e26d36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wq8pj\" (UID: \"f599311f-5832-43e6-9aa3-342bf7e26d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.321887 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.321869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f599311f-5832-43e6-9aa3-342bf7e26d36-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wq8pj\" (UID: \"f599311f-5832-43e6-9aa3-342bf7e26d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.329701 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.329676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8jcr\" (UniqueName: \"kubernetes.io/projected/f599311f-5832-43e6-9aa3-342bf7e26d36-kube-api-access-t8jcr\") pod \"kuadrant-operator-controller-manager-5769f99d9f-wq8pj\" (UID: \"f599311f-5832-43e6-9aa3-342bf7e26d36\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.417252 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.417188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:35.693683 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:35.693651    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"]
Sep 14 08:27:35.697260 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:27:35.697232    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf599311f_5832_43e6_9aa3_342bf7e26d36.slice/crio-a96eb777e04254053dda3771662e6a040964ceb754b93a287360f8e8e73d5bc8 WatchSource:0}: Error finding container a96eb777e04254053dda3771662e6a040964ceb754b93a287360f8e8e73d5bc8: Status 404 returned error can't find the container with id a96eb777e04254053dda3771662e6a040964ceb754b93a287360f8e8e73d5bc8
Sep 14 08:27:36.530352 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:36.530309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj" event={"ID":"f599311f-5832-43e6-9aa3-342bf7e26d36","Type":"ContainerStarted","Data":"9b0aca9a20afc605547d833a90632ad65987f9e24d4f2edc90cdebc7b4ed19e7"}
Sep 14 08:27:36.530812 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:36.530371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj" event={"ID":"f599311f-5832-43e6-9aa3-342bf7e26d36","Type":"ContainerStarted","Data":"a96eb777e04254053dda3771662e6a040964ceb754b93a287360f8e8e73d5bc8"}
Sep 14 08:27:36.530812 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:36.530478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:27:36.551206 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:36.551159    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj" podStartSLOduration=1.551143358 podStartE2EDuration="1.551143358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:27:36.54899768 +0000 UTC m=+2312.257011819" watchObservedRunningTime="2026-09-14 08:27:36.551143358 +0000 UTC m=+2312.259157498"
Sep 14 08:27:47.537653 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:27:47.537619    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-wq8pj"
Sep 14 08:28:05.369432 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:05.369352    2563 scope.go:117] "RemoveContainer" containerID="b4edb8d79e7074fb8819d3de293ee57a08ddfeedf5f7d71260b43bf2b9cb7c6c"
Sep 14 08:28:05.379237 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:05.379217    2563 scope.go:117] "RemoveContainer" containerID="35b99d4fcca08d31299995c1b55d766176cb7a10ab8da45ef8830253d0bdfc1e"
Sep 14 08:28:31.281587 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.281554    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"]
Sep 14 08:28:31.285170 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.285145    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.287603 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.287582    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-7066a4c2-kserve-self-signed-certs\""
Sep 14 08:28:31.292272 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.291867    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"]
Sep 14 08:28:31.395967 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.395931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-home\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.396155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.396002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-model-cache\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.396155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.396028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tmp-dir\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.396155 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.396075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbj4d\" (UniqueName: \"kubernetes.io/projected/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-kube-api-access-hbj4d\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.396298 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.396162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tls-certs\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.396298 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.396191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-dshm\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.496795 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.496762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tmp-dir\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.496795 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.496805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbj4d\" (UniqueName: \"kubernetes.io/projected/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-kube-api-access-hbj4d\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.496847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tls-certs\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.496979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-dshm\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497156 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.497044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-home\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497217 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.497184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-model-cache\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497217 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.497192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tmp-dir\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497410 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.497391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-home\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.497497 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.497479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-model-cache\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.499143 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.499120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-dshm\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.499347 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.499329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tls-certs\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.504416 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.504398    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbj4d\" (UniqueName: \"kubernetes.io/projected/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-kube-api-access-hbj4d\") pod \"e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") " pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.594668 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.594601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:31.869043 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:31.869014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"]
Sep 14 08:28:31.871440 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:28:31.871413    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a604c00_0e1a_4aea_95cf_ca8b9754c46a.slice/crio-da1475abb03b3e184962ec1eb0668f5d41f045d5e9c79ed4e9254c60fd4a1162 WatchSource:0}: Error finding container da1475abb03b3e184962ec1eb0668f5d41f045d5e9c79ed4e9254c60fd4a1162: Status 404 returned error can't find the container with id da1475abb03b3e184962ec1eb0668f5d41f045d5e9c79ed4e9254c60fd4a1162
Sep 14 08:28:32.834483 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:32.834448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4" event={"ID":"1a604c00-0e1a-4aea-95cf-ca8b9754c46a","Type":"ContainerStarted","Data":"ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4"}
Sep 14 08:28:32.834483 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:32.834486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4" event={"ID":"1a604c00-0e1a-4aea-95cf-ca8b9754c46a","Type":"ContainerStarted","Data":"da1475abb03b3e184962ec1eb0668f5d41f045d5e9c79ed4e9254c60fd4a1162"}
Sep 14 08:28:32.850481 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:32.850433    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4" podStartSLOduration=1.850419992 podStartE2EDuration="1.850419992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:28:32.849320856 +0000 UTC m=+2368.557334996" watchObservedRunningTime="2026-09-14 08:28:32.850419992 +0000 UTC m=+2368.558434130"
Sep 14 08:28:41.595692 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:41.595651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:41.595692 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:41.595700    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:41.607935 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:41.607912    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:28:41.890905 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:28:41.890831    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:29:04.925277 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:04.925248    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:29:04.930428 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:04.930401    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:29:06.382152 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:06.382122    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"]
Sep 14 08:29:21.402114 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.402071    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4" podUID="1a604c00-0e1a-4aea-95cf-ca8b9754c46a" containerName="main" containerID="cri-o://ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4" gracePeriod=45
Sep 14 08:29:21.643050 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.642982    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:29:21.765718 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765687    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tmp-dir\") pod \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") "
Sep 14 08:29:21.765913 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765756    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbj4d\" (UniqueName: \"kubernetes.io/projected/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-kube-api-access-hbj4d\") pod \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") "
Sep 14 08:29:21.765913 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-home\") pod \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") "
Sep 14 08:29:21.765913 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tls-certs\") pod \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") "
Sep 14 08:29:21.765913 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765845    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-model-cache\") pod \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") "
Sep 14 08:29:21.765913 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765861    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-dshm\") pod \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\" (UID: \"1a604c00-0e1a-4aea-95cf-ca8b9754c46a\") "
Sep 14 08:29:21.766193 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.765949    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a604c00-0e1a-4aea-95cf-ca8b9754c46a" (UID: "1a604c00-0e1a-4aea-95cf-ca8b9754c46a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:21.766193 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.766104    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a604c00-0e1a-4aea-95cf-ca8b9754c46a" (UID: "1a604c00-0e1a-4aea-95cf-ca8b9754c46a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:21.766193 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.766121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-home" (OuterVolumeSpecName: "home") pod "1a604c00-0e1a-4aea-95cf-ca8b9754c46a" (UID: "1a604c00-0e1a-4aea-95cf-ca8b9754c46a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:21.766310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.766214    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tmp-dir\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:29:21.766310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.766239    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-home\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:29:21.766310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.766255    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-model-cache\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:29:21.768083 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.768058    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a604c00-0e1a-4aea-95cf-ca8b9754c46a" (UID: "1a604c00-0e1a-4aea-95cf-ca8b9754c46a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:29:21.768177 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.768130    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-dshm" (OuterVolumeSpecName: "dshm") pod "1a604c00-0e1a-4aea-95cf-ca8b9754c46a" (UID: "1a604c00-0e1a-4aea-95cf-ca8b9754c46a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:29:21.768177 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.768138    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-kube-api-access-hbj4d" (OuterVolumeSpecName: "kube-api-access-hbj4d") pod "1a604c00-0e1a-4aea-95cf-ca8b9754c46a" (UID: "1a604c00-0e1a-4aea-95cf-ca8b9754c46a"). InnerVolumeSpecName "kube-api-access-hbj4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:29:21.867338 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.867298    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-tls-certs\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:29:21.867338 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.867329    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-dshm\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:29:21.867338 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:21.867342    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbj4d\" (UniqueName: \"kubernetes.io/projected/1a604c00-0e1a-4aea-95cf-ca8b9754c46a-kube-api-access-hbj4d\") on node \"ip-10-0-135-172.ec2.internal\" DevicePath \"\""
Sep 14 08:29:22.085340 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.085258    2563 generic.go:356] "Generic (PLEG): container finished" podID="1a604c00-0e1a-4aea-95cf-ca8b9754c46a" containerID="ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4" exitCode=0
Sep 14 08:29:22.085340 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.085305    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"
Sep 14 08:29:22.085340 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.085314    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4" event={"ID":"1a604c00-0e1a-4aea-95cf-ca8b9754c46a","Type":"ContainerDied","Data":"ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4"}
Sep 14 08:29:22.085562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.085354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4" event={"ID":"1a604c00-0e1a-4aea-95cf-ca8b9754c46a","Type":"ContainerDied","Data":"da1475abb03b3e184962ec1eb0668f5d41f045d5e9c79ed4e9254c60fd4a1162"}
Sep 14 08:29:22.085562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.085372    2563 scope.go:117] "RemoveContainer" containerID="ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4"
Sep 14 08:29:22.094595 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.094578    2563 scope.go:117] "RemoveContainer" containerID="ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4"
Sep 14 08:29:22.094921 ip-10-0-135-172 kubenswrapper[2563]: E0914 08:29:22.094888    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4\": container with ID starting with ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4 not found: ID does not exist" containerID="ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4"
Sep 14 08:29:22.095004 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.094928    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4"} err="failed to get container status \"ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4\": rpc error: code = NotFound desc = could not find container \"ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4\": container with ID starting with ea895ac304cb6cfcbdfb92bf013fea7a07d93d8f3cd157829b8d2939cdadb5b4 not found: ID does not exist"
Sep 14 08:29:22.106731 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.106699    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"]
Sep 14 08:29:22.109951 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.109929    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-7066a4c2-kserve-6d5cf6447-vvbv4"]
Sep 14 08:29:22.789623 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:22.789580    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a604c00-0e1a-4aea-95cf-ca8b9754c46a" path="/var/lib/kubelet/pods/1a604c00-0e1a-4aea-95cf-ca8b9754c46a/volumes"
Sep 14 08:29:34.521256 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.521164    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"]
Sep 14 08:29:34.522631 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.522601    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a604c00-0e1a-4aea-95cf-ca8b9754c46a" containerName="main"
Sep 14 08:29:34.522806 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.522793    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a604c00-0e1a-4aea-95cf-ca8b9754c46a" containerName="main"
Sep 14 08:29:34.523127 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.523110    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a604c00-0e1a-4aea-95cf-ca8b9754c46a" containerName="main"
Sep 14 08:29:34.529816 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.529788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.532149 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.532086    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 08:29:34.532846 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.532821    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"]
Sep 14 08:29:34.533215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.533193    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:29:34.533215 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.533209    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zwz44\""
Sep 14 08:29:34.533368 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.533209    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:29:34.533368 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.533200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:29:34.584296 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.584266    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9455\" (UniqueName: \"kubernetes.io/projected/99e2b215-a5c3-469a-bdf2-0444d3ebd636-kube-api-access-s9455\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.584418 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.584308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99e2b215-a5c3-469a-bdf2-0444d3ebd636-metrics-certs\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.584418 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.584338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e2b215-a5c3-469a-bdf2-0444d3ebd636-openshift-service-ca\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.584498 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.584448    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e2b215-a5c3-469a-bdf2-0444d3ebd636-maas-api-tls\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.685100 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.685070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e2b215-a5c3-469a-bdf2-0444d3ebd636-maas-api-tls\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.685239 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.685121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9455\" (UniqueName: \"kubernetes.io/projected/99e2b215-a5c3-469a-bdf2-0444d3ebd636-kube-api-access-s9455\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.685239 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.685153    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99e2b215-a5c3-469a-bdf2-0444d3ebd636-metrics-certs\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.685239 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.685182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e2b215-a5c3-469a-bdf2-0444d3ebd636-openshift-service-ca\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.685827 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.685806    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99e2b215-a5c3-469a-bdf2-0444d3ebd636-openshift-service-ca\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.687623 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.687601    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99e2b215-a5c3-469a-bdf2-0444d3ebd636-metrics-certs\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.687706 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.687613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99e2b215-a5c3-469a-bdf2-0444d3ebd636-maas-api-tls\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.692925 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.692905    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9455\" (UniqueName: \"kubernetes.io/projected/99e2b215-a5c3-469a-bdf2-0444d3ebd636-kube-api-access-s9455\") pod \"maas-api-579fddb5f-5zgkz\" (UID: \"99e2b215-a5c3-469a-bdf2-0444d3ebd636\") " pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:34.843662 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:34.843583    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:35.119797 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:35.116778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"]
Sep 14 08:29:35.120392 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:29:35.120360    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e2b215_a5c3_469a_bdf2_0444d3ebd636.slice/crio-64f974eb07478c8fa3cecb25b7307f6a36dd5ad48adb80be75427138505825e6 WatchSource:0}: Error finding container 64f974eb07478c8fa3cecb25b7307f6a36dd5ad48adb80be75427138505825e6: Status 404 returned error can't find the container with id 64f974eb07478c8fa3cecb25b7307f6a36dd5ad48adb80be75427138505825e6
Sep 14 08:29:35.122064 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:35.122034    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:29:35.152866 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:35.152843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz" event={"ID":"99e2b215-a5c3-469a-bdf2-0444d3ebd636","Type":"ContainerStarted","Data":"64f974eb07478c8fa3cecb25b7307f6a36dd5ad48adb80be75427138505825e6"}
Sep 14 08:29:36.160479 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:36.160444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz" event={"ID":"99e2b215-a5c3-469a-bdf2-0444d3ebd636","Type":"ContainerStarted","Data":"302894f009183015b8076948f81e6432ddf188b67b261e31e26782fd5a266e0b"}
Sep 14 08:29:36.160986 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:36.160596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:36.180311 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:36.180267    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz" podStartSLOduration=1.842225271 podStartE2EDuration="2.18025456s" totalImagesPullingTime="338.029289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:29:36.179228776 +0000 UTC m=+2431.887242916" watchObservedRunningTime="2026-09-14 08:29:36.18025456 +0000 UTC m=+2431.888268715"
Sep 14 08:29:42.171694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:42.171660    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-579fddb5f-5zgkz"
Sep 14 08:29:54.739488 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:29:54.739451    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cb94df66-5sgr9_6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6/authorino/0.log"
Sep 14 08:30:01.094293 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:01.094257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-bhn5g_508738a5-48cb-4f6f-8e5b-3adc538d429f/copy-manifests/0.log"
Sep 14 08:30:01.103011 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:01.102981    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-bhn5g_508738a5-48cb-4f6f-8e5b-3adc538d429f/manager/0.log"
Sep 14 08:30:01.716392 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:01.716366    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-579fddb5f-5zgkz_99e2b215-a5c3-469a-bdf2-0444d3ebd636/maas-api/0.log"
Sep 14 08:30:01.948585 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:01.948560    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822880-mc46v_7b4c9cb9-69a1-489c-875e-75d6d3e7e63f/cleanup/0.log"
Sep 14 08:30:02.058516 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:02.058441    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822895-5zszm_153c7cb9-3dd1-4979-ac8e-63324b2670f1/cleanup/0.log"
Sep 14 08:30:03.099944 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:03.099891    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cb94df66-5sgr9_6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6/authorino/0.log"
Sep 14 08:30:03.578812 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:03.578780    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-wq8pj_f599311f-5832-43e6-9aa3-342bf7e26d36/manager/0.log"
Sep 14 08:30:04.530758 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:04.530645    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-kkqw4_6b2624e5-3340-4603-9c45-f44e3915cb8a/kube-auth-proxy/0.log"
Sep 14 08:30:05.399015 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:05.398983    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4vpr6_441dbbb3-71f8-4207-bfe3-9497cd26c015/main/0.log"
Sep 14 08:30:05.412789 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:05.412764    2563 scope.go:117] "RemoveContainer" containerID="1a1a505607521c1a6d607e353a0085663eaa1b5b78000d637e12dba2c4b51118"
Sep 14 08:30:05.422086 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:05.422067    2563 scope.go:117] "RemoveContainer" containerID="f71036a03217e081843edc70e93c22c8eea13641ad2593a750cfcc97fb4fdbb5"
Sep 14 08:30:05.429417 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:05.429402    2563 scope.go:117] "RemoveContainer" containerID="e6fcca6618f60a98b4bc53af2dc470cbd61a3a85d82e648136b030c75f5fa934"
Sep 14 08:30:05.644110 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:05.644077    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-hzlrn_1cf1d9b0-2a8f-4b12-b444-62abc2a7f897/main/0.log"
Sep 14 08:30:05.998454 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:05.998427    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-k6r4w_62d17039-8bd6-4344-8d20-60affee6097f/main/0.log"
Sep 14 08:30:06.115658 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:06.115628    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-cs2qr_f4e18741-812b-4a12-b455-9ae54a4f1ef1/main/0.log"
Sep 14 08:30:15.243364 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:15.243335    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cxxtl_0f884ddb-b97a-4fe4-bd3a-a0aaeb6e0d24/global-pull-secret-syncer/0.log"
Sep 14 08:30:15.429064 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:15.429036    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xdppf_fba2bd14-caf9-4175-a048-158d4665646a/konnectivity-agent/0.log"
Sep 14 08:30:15.491704 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:15.491678    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-172.ec2.internal_2ac03057413443baa8a950301b2f270d/haproxy/0.log"
Sep 14 08:30:19.541448 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:19.541417    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cb94df66-5sgr9_6d629ada-6ee5-4dc1-a8ee-b8e79e14edb6/authorino/0.log"
Sep 14 08:30:19.686045 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:19.686016    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-wq8pj_f599311f-5832-43e6-9aa3-342bf7e26d36/manager/0.log"
Sep 14 08:30:21.322234 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.322154    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/alertmanager/0.log"
Sep 14 08:30:21.343411 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.343387    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/config-reloader/0.log"
Sep 14 08:30:21.364196 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.364174    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/kube-rbac-proxy-web/0.log"
Sep 14 08:30:21.382608 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.382581    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/kube-rbac-proxy/0.log"
Sep 14 08:30:21.401562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.401538    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/kube-rbac-proxy-metric/0.log"
Sep 14 08:30:21.418901 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.418879    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/prom-label-proxy/0.log"
Sep 14 08:30:21.441799 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.441779    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_b6f5eaa2-1c99-473e-b65e-74a6a2c25f9d/init-config-reloader/0.log"
Sep 14 08:30:21.769311 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.769284    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8t8p_5ad65499-1979-4585-b1da-a643541ee2ac/node-exporter/0.log"
Sep 14 08:30:21.788633 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.788606    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8t8p_5ad65499-1979-4585-b1da-a643541ee2ac/kube-rbac-proxy/0.log"
Sep 14 08:30:21.808390 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.808369    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8t8p_5ad65499-1979-4585-b1da-a643541ee2ac/init-textfile/0.log"
Sep 14 08:30:21.840432 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.840375    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-djrd2_e2b3c120-cc4b-4797-90f8-d6233738cc21/kube-rbac-proxy-main/0.log"
Sep 14 08:30:21.858022 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.858005    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-djrd2_e2b3c120-cc4b-4797-90f8-d6233738cc21/kube-rbac-proxy-self/0.log"
Sep 14 08:30:21.878720 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.878697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-djrd2_e2b3c120-cc4b-4797-90f8-d6233738cc21/openshift-state-metrics/0.log"
Sep 14 08:30:21.932541 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.932524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/prometheus/0.log"
Sep 14 08:30:21.950052 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.950038    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/config-reloader/0.log"
Sep 14 08:30:21.968817 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.968800    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/thanos-sidecar/0.log"
Sep 14 08:30:21.988877 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:21.988863    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/kube-rbac-proxy-web/0.log"
Sep 14 08:30:22.009562 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.009543    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/kube-rbac-proxy/0.log"
Sep 14 08:30:22.029240 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.029224    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/kube-rbac-proxy-thanos/0.log"
Sep 14 08:30:22.051415 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.051399    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d66a7e8c-1e67-4e42-b388-2cf41495f5cc/init-config-reloader/0.log"
Sep 14 08:30:22.134629 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.134555    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-8fkqq_b2f954db-a555-4515-8dd8-29492c52b79b/prometheus-operator-admission-webhook/0.log"
Sep 14 08:30:22.161775 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.161751    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8df564599-jc6ng_48a63f12-7df7-45eb-b81b-cc1126db9bba/telemeter-client/0.log"
Sep 14 08:30:22.185631 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.185610    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8df564599-jc6ng_48a63f12-7df7-45eb-b81b-cc1126db9bba/reload/0.log"
Sep 14 08:30:22.207679 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.207660    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8df564599-jc6ng_48a63f12-7df7-45eb-b81b-cc1126db9bba/kube-rbac-proxy/0.log"
Sep 14 08:30:22.238457 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.238437    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5f4b76bb-ns66k_d5f7c5c6-cc5c-4423-a609-ccfb30f837ac/thanos-query/0.log"
Sep 14 08:30:22.260932 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.260912    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5f4b76bb-ns66k_d5f7c5c6-cc5c-4423-a609-ccfb30f837ac/kube-rbac-proxy-web/0.log"
Sep 14 08:30:22.286640 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.286620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5f4b76bb-ns66k_d5f7c5c6-cc5c-4423-a609-ccfb30f837ac/kube-rbac-proxy/0.log"
Sep 14 08:30:22.325495 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.325472    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5f4b76bb-ns66k_d5f7c5c6-cc5c-4423-a609-ccfb30f837ac/prom-label-proxy/0.log"
Sep 14 08:30:22.341710 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.341692    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5f4b76bb-ns66k_d5f7c5c6-cc5c-4423-a609-ccfb30f837ac/kube-rbac-proxy-rules/0.log"
Sep 14 08:30:22.366029 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:22.366013    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-b5f4b76bb-ns66k_d5f7c5c6-cc5c-4423-a609-ccfb30f837ac/kube-rbac-proxy-metrics/0.log"
Sep 14 08:30:24.501802 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:24.501772    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/1.log"
Sep 14 08:30:24.507118 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:24.507097    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-f9dtl_71927f5a-bf06-4703-9470-8398d43031b0/console-operator/2.log"
Sep 14 08:30:25.050735 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.050698    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c9f6f8d89-xcgpz_1151113c-0ea2-4dcd-9586-ae829213fb09/console/0.log"
Sep 14 08:30:25.078589 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.078561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-fblrm_9fe2ba75-cb96-46a8-8a04-321885ea1cc7/download-server/0.log"
Sep 14 08:30:25.788413 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.788379    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"]
Sep 14 08:30:25.792182 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.792160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.794716 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.794691    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bqggw\"/\"kube-root-ca.crt\""
Sep 14 08:30:25.795610 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.795585    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bqggw\"/\"default-dockercfg-frvbt\""
Sep 14 08:30:25.795694 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.795615    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bqggw\"/\"openshift-service-ca.crt\""
Sep 14 08:30:25.801987 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.801965    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"]
Sep 14 08:30:25.860698 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.860674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-sys\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.860816 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.860750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8lnx\" (UniqueName: \"kubernetes.io/projected/f7bc538c-483d-4b28-a5f5-36c41448c3a0-kube-api-access-t8lnx\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.860816 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.860800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-proc\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.860890 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.860818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-podres\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.860926 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.860908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-lib-modules\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962346 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-proc\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962486 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-podres\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962486 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-lib-modules\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962486 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-sys\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962486 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962449    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-proc\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962493    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-podres\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-sys\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962561    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8lnx\" (UniqueName: \"kubernetes.io/projected/f7bc538c-483d-4b28-a5f5-36c41448c3a0-kube-api-access-t8lnx\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.962656 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.962566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7bc538c-483d-4b28-a5f5-36c41448c3a0-lib-modules\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:25.970429 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:25.970410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8lnx\" (UniqueName: \"kubernetes.io/projected/f7bc538c-483d-4b28-a5f5-36c41448c3a0-kube-api-access-t8lnx\") pod \"perf-node-gather-daemonset-c8n5f\" (UID: \"f7bc538c-483d-4b28-a5f5-36c41448c3a0\") " pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:26.103111 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:26.103038    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:26.371809 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:26.371786    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"]
Sep 14 08:30:26.373614 ip-10-0-135-172 kubenswrapper[2563]: W0914 08:30:26.373587    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7bc538c_483d_4b28_a5f5_36c41448c3a0.slice/crio-e245696b4433c279dca09bbca41177a91130257816c3fd33a6ac3fbe48ab5bad WatchSource:0}: Error finding container e245696b4433c279dca09bbca41177a91130257816c3fd33a6ac3fbe48ab5bad: Status 404 returned error can't find the container with id e245696b4433c279dca09bbca41177a91130257816c3fd33a6ac3fbe48ab5bad
Sep 14 08:30:26.418578 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:26.418550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f" event={"ID":"f7bc538c-483d-4b28-a5f5-36c41448c3a0","Type":"ContainerStarted","Data":"e245696b4433c279dca09bbca41177a91130257816c3fd33a6ac3fbe48ab5bad"}
Sep 14 08:30:26.964613 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:26.964579    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9bdwp_af1713b8-c170-4d6b-a8d9-b53309b2ac75/dns/0.log"
Sep 14 08:30:26.985845 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:26.985823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9bdwp_af1713b8-c170-4d6b-a8d9-b53309b2ac75/kube-rbac-proxy/0.log"
Sep 14 08:30:27.095211 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:27.095187    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j786z_fb587b36-5ad8-4a6e-bce6-9d00228fdef6/dns-node-resolver/0.log"
Sep 14 08:30:27.425470 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:27.425387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f" event={"ID":"f7bc538c-483d-4b28-a5f5-36c41448c3a0","Type":"ContainerStarted","Data":"e5663b17ae0eaccb14fe42ec83ee157353472d7a12c362607252673ddd3e6bc4"}
Sep 14 08:30:27.425603 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:27.425510    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:27.444509 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:27.444466    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f" podStartSLOduration=2.444453068 podStartE2EDuration="2.444453068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 08:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:30:27.443311133 +0000 UTC m=+2483.151325272" watchObservedRunningTime="2026-09-14 08:30:27.444453068 +0000 UTC m=+2483.152467206"
Sep 14 08:30:27.697466 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:27.697436    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2jjqz_32354ba1-68bb-449a-88b5-629a449b78be/node-ca/0.log"
Sep 14 08:30:29.069862 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:29.069830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-kkqw4_6b2624e5-3340-4603-9c45-f44e3915cb8a/kube-auth-proxy/0.log"
Sep 14 08:30:30.122682 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:30.122652    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rzkr2_97683ac6-241b-47bc-b93d-2b921a7f7140/serve-healthcheck-canary/0.log"
Sep 14 08:30:30.572542 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:30.572511    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-8lgjz_b8bae1e3-c38a-4e9a-a6a2-e4625a779de4/insights-operator/0.log"
Sep 14 08:30:30.572764 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:30.572744    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-8lgjz_b8bae1e3-c38a-4e9a-a6a2-e4625a779de4/insights-operator/1.log"
Sep 14 08:30:30.590557 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:30.590534    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2vgb9_f2e94682-b81a-42e1-b37d-58b15daa351c/kube-rbac-proxy/0.log"
Sep 14 08:30:30.608310 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:30.608289    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2vgb9_f2e94682-b81a-42e1-b37d-58b15daa351c/exporter/0.log"
Sep 14 08:30:30.627284 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:30.627266    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2vgb9_f2e94682-b81a-42e1-b37d-58b15daa351c/extractor/0.log"
Sep 14 08:30:33.290738 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:33.290691    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-bhn5g_508738a5-48cb-4f6f-8e5b-3adc538d429f/manager/0.log"
Sep 14 08:30:33.309572 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:33.309550    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4cb54c55-bhn5g_508738a5-48cb-4f6f-8e5b-3adc538d429f/copy-manifests/0.log"
Sep 14 08:30:33.440397 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:33.440372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bqggw/perf-node-gather-daemonset-c8n5f"
Sep 14 08:30:34.676524 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:34.676456    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b895dd7c5-mmf9j_e7762b50-008f-4a2f-855a-bbb9c76fc036/manager/0.log"
Sep 14 08:30:34.727369 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:34.727342    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kdcgn_28324170-1ef9-418d-a5a2-6d98c5b387d9/openshift-lws-operator/0.log"
Sep 14 08:30:36.502608 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:36.502579    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-kkqw4_6b2624e5-3340-4603-9c45-f44e3915cb8a/kube-auth-proxy/0.log"
Sep 14 08:30:38.620840 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:38.620812    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fb8dbff8-kkqw4_6b2624e5-3340-4603-9c45-f44e3915cb8a/kube-auth-proxy/0.log"
Sep 14 08:30:41.577817 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.577790    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/kube-multus-additional-cni-plugins/0.log"
Sep 14 08:30:41.595807 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.595783    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/egress-router-binary-copy/0.log"
Sep 14 08:30:41.614197 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.614176    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/cni-plugins/0.log"
Sep 14 08:30:41.631874 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.631855    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/bond-cni-plugin/0.log"
Sep 14 08:30:41.649394 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.649376    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/routeoverride-cni/0.log"
Sep 14 08:30:41.667223 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.667208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/whereabouts-cni-bincopy/0.log"
Sep 14 08:30:41.691361 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:41.691343    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5gctk_21b8f02f-945b-4f0b-84bf-98a0053d7f18/whereabouts-cni/0.log"
Sep 14 08:30:42.114669 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:42.114640    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xtts5_be159616-10cd-4bbc-9db6-dccb069099cd/kube-multus/0.log"
Sep 14 08:30:42.134822 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:42.134798    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7xmvb_223808bc-d985-4ead-8463-c63a3972b62b/network-metrics-daemon/0.log"
Sep 14 08:30:42.148775 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:42.148756    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7xmvb_223808bc-d985-4ead-8463-c63a3972b62b/kube-rbac-proxy/0.log"
Sep 14 08:30:43.259043 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:43.259011    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hl96f_748bdf6e-1c14-4269-b408-6fac8c483071/ovn-controller/0.log"
Sep 14 08:30:43.275282 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:43.275255    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hl96f_748bdf6e-1c14-4269-b408-6fac8c483071/ovn-acl-logging/0.log"
Sep 14 08:30:43.299633 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:43.299606    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hl96f_748bdf6e-1c14-4269-b408-6fac8c483071/kube-rbac-proxy-node/0.log"
Sep 14 08:30:43.320961 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:43.320936    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hl96f_748bdf6e-1c14-4269-b408-6fac8c483071/kube-rbac-proxy-ovn-metrics/0.log"
Sep 14 08:30:43.338448 ip-10-0-135-172 kubenswrapper[2563]: I0914 08:30:43.338426    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hl96f_748bdf6e-1c14-4269-b408-6fac8c483071/northd/0.log"