Sep 10 11:49:48.832247 ip-10-0-129-41 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 11:49:48.832256 ip-10-0-129-41 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 11:49:48.832263 ip-10-0-129-41 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 11:49:48.832463 ip-10-0-129-41 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 11:49:58.906197 ip-10-0-129-41 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 11:49:58.906230 ip-10-0-129-41 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 404d7c4b4abc40bdb5241083d149fa47 --
Sep 10 11:51:28.232174 ip-10-0-129-41 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 11:51:28.683639 ip-10-0-129-41 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 11:51:28.683639 ip-10-0-129-41 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 11:51:28.683639 ip-10-0-129-41 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 11:51:28.683639 ip-10-0-129-41 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 11:51:28.683639 ip-10-0-129-41 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 11:51:28.684683 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.684589    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 11:51:28.688722 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688704    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:51:28.688722 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688722    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688726    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688731    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688735    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688738    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688741    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688744    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688747    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688753    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688756    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688760    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688763    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688766    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688768    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688771    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688774    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688777    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688780    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688782    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688785    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:51:28.688785 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688787    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688790    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688793    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688796    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688799    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688802    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688805    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688808    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688810    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688814    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688818    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688821    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688824    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688827    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688830    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688833    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688835    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688837    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688840    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688842    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:51:28.689286 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688845    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688848    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688850    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688870    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688874    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688878    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688882    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688886    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688889    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688891    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688893    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688896    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688899    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688901    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688904    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688907    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688910    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688913    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688915    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688918    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:51:28.689780 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688920    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688923    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688925    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688929    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688932    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688935    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688937    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688940    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688943    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688946    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688949    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688952    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688954    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688957    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688959    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688962    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688964    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688967    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688969    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:51:28.690293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688971    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688974    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688977    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688979    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688982    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688984    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688986    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688990    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688995    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.688998    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689001    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689004    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689008    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689011    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689014    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689017    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689020    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689023    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689025    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:51:28.690853 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689746    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689755    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689758    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689761    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689764    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689767    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689770    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689772    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689775    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689778    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689781    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689784    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689786    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689789    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689792    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689795    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689798    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689800    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689803    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689805    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:51:28.691337 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689809    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689812    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689815    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689818    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689820    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689823    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689827    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689831    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689834    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689836    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689839    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689842    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689844    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689847    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689849    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689852    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689869    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689872    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689875    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:51:28.691879 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689893    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689897    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689900    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689903    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689906    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689909    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689911    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689914    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689916    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689919    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689922    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689924    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689927    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689930    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689933    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689936    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689938    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689941    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689946    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:51:28.692355 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689950    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689953    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689956    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689959    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689962    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689964    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689967    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689970    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689972    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689975    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689977    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689980    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689982    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689986    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689988    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689991    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689993    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689996    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.689999    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690002    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:51:28.692830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690004    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690007    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690009    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690012    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690015    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690017    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690020    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690023    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690026    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690029    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690032    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690034    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690037    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690040    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690044    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690048    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690051    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690054    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690057    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690059    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:51:28.693340 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690062    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690430    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690435    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690438    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690441    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690444    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690447    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690450    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690452    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690455    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690458    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690460    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690463    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690465    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690469    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690473    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690476    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690479    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690482    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:51:28.693830 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690486    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690488    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690491    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690494    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690497    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690500    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690502    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690505    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690508    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690510    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690514    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690517    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690520    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690523    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690526    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690529    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690532    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690535    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690538    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690541    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:51:28.694313 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690544    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690547    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690549    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690552    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690555    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690557    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690559    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690562    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690565    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690567    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690571    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690574    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690577    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690580    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690582    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690585    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690588    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690591    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690593    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:51:28.694814 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690596    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690598    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690601    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690604    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690606    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690609    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690611    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690614    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690616    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690619    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690621    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690624    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690626    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690638    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690641    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690644    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690647    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690650    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690652    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690655    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:51:28.695295 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690659    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690662    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690666    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690669    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690671    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690675    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690678    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690680    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690683    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690686    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690688    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690691    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690693    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690696    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690698    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690701    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690706    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690708    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690711    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:51:28.695775 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690713    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690716    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.690718    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690801    2571 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690849    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690868    2571 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690874    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690879    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690882    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690887    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690891    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690894    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690897    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690901    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690904    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690907    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690910    2571 flags.go:64] FLAG: --cgroup-root=""
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690914    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690917    2571 flags.go:64] FLAG: --client-ca-file=""
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690920    2571 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690923    2571 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690929    2571 flags.go:64] FLAG: --cluster-domain=""
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690932    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690935    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 11:51:28.696255 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690938    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690941    2571 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690945    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690948    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690951    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690956    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690960    2571 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690963    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690966    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690969    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690972    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690976    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690979    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690982    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690985    2571 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690988    2571 flags.go:64] FLAG: --enable-server="true"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690992    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690996    2571 flags.go:64] FLAG: --event-burst="100"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.690999    2571 flags.go:64] FLAG: --event-qps="50"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691002    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691005    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691008    2571 flags.go:64] FLAG: --eviction-hard=""
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691012    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691015    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691017    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 11:51:28.696877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691020    2571 flags.go:64] FLAG: --eviction-soft=""
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691023    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691026    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691031    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691034    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691038    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691041    2571 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691043    2571 flags.go:64] FLAG: --feature-gates=""
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691047    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691050    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691054    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691057    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691061    2571 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691065    2571 flags.go:64] FLAG: --help="false"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691068    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691071    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691075    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691078    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691081    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691084    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691088    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691091    2571 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691093    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 11:51:28.697482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691096    2571 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691100    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691103    2571 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691106    2571 flags.go:64] FLAG: --kube-reserved=""
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691109    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691112    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691115    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691118    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691121    2571 flags.go:64] FLAG: --lock-file=""
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691124    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691126    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691130    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691136    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691139    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691142    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691145    2571 flags.go:64] FLAG: --logging-format="text"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691148    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691151    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691154    2571 flags.go:64] FLAG: --manifest-url=""
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691157    2571 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691161    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691166    2571 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691170    2571 flags.go:64] FLAG: --max-pods="110"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691173    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691176    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 11:51:28.698128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691179    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691182    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691185    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691188    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691191    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691198    2571 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691201    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691204    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691207    2571 flags.go:64] FLAG: --pod-cidr=""
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691211    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691217    2571 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691220    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691223    2571 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691226    2571 flags.go:64] FLAG: --port="10250"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691229    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691232    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ec7978dd76a3d823"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691235    2571 flags.go:64] FLAG: --qos-reserved=""
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691238    2571 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691242    2571 flags.go:64] FLAG: --register-node="true"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691246    2571 flags.go:64] FLAG: --register-with-taints=""
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691249    2571 flags.go:64] FLAG: --registry-burst="10"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691252    2571 flags.go:64] FLAG: --registry-qps="5"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691255    2571 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691258    2571 flags.go:64] FLAG: --reserved-memory=""
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691262    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 11:51:28.698843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691265    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691268    2571 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691271    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691274    2571 flags.go:64] FLAG: --runonce="false"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691278    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691281    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691284    2571 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691287    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691290    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691293    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691296    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691299    2571 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691302    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691305    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691307    2571 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691310    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691313    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691317    2571 flags.go:64] FLAG: --system-cgroups=""
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691320    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691325    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691328    2571 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691331    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691335    2571 flags.go:64] FLAG: --tls-min-version=""
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691338    2571 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691341    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 11:51:28.699943 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691344    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691349    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691352    2571 flags.go:64] FLAG: --v="2"
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691356    2571 flags.go:64] FLAG: --version="false"
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691360    2571 flags.go:64] FLAG: --vmodule=""
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691363    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.691367    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691484    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691487    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691490    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691494    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691498    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691500    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691503    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691505    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691508    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691511    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691513    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691516    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691519    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691521    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:51:28.701082 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691524    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691527    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691529    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691532    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691536    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691538    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691541    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691543    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691546    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691549    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691552    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691555    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691559    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691561    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691564    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691566    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691569    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691571    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691576    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691580    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:51:28.702103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691583    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691586    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691589    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691592    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691595    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691598    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691601    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691603    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691606    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691609    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691612    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691615    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691617    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691621    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691624    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691627    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691629    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691633    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691636    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:51:28.703006 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691639    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691642    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691644    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691647    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691650    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691655    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691658    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691660    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691663    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691665    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691668    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691670    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691673    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691676    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691678    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691681    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691684    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691687    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691689    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691692    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:51:28.703520 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691694    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691697    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691701    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691704    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691707    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691709    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691712    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691714    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691717    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691719    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691722    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691725    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691729    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691733    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691737    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691741    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691744    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691748    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691751    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691754    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:51:28.704309 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691756    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691758    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691761    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691763    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691766    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.691769    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.692429    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.701382    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.701405    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.701432    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.701444    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701537    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701545    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701549    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701553    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:51:28.705168 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701558    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701561    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701568    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701575    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701580    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701586    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701593    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701598    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701603    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701607    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701612    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701616    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701620    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701624    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701628    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701632    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701636    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701640    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701644    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:51:28.705570 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701648    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701653    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701657    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701661    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701665    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701669    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701682    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701687    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701691    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701696    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701700    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701704    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701708    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701712    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701716    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701720    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701725    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701729    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701733    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:51:28.706220 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701737    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701741    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701746    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701750    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701754    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701758    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701762    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701766    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701770    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701775    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701779    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701783    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701787    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701791    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701795    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701798    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701803    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701807    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701811    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701815    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:51:28.706733 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701827    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701832    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701836    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701840    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701844    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701848    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701871    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701877    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701881    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701885    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701890    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701894    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701898    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701901    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701905    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701909    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701913    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701917    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701921    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701924    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:51:28.707320 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701928    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701932    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701936    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701940    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701945    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701949    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701953    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701957    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701961    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701965    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701969    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701974    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701978    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701991    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701995    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.701999    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:51:28.707828 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702003    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.702011    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702241    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702250    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702254    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702258    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702263    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702267    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702272    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702275    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702280    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702284    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702291    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702296    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702301    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:51:28.708271 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702306    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702311    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702315    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702327    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702331    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702335    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702339    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702343    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702348    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702352    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702356    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702360    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702365    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702370    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702374    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702387    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702392    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702396    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702400    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702405    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:51:28.708659 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702409    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702413    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702417    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702421    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702425    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702429    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702433    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702437    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702442    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702446    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702450    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702454    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702457    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702462    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702466    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702470    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702474    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702478    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702482    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702486    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:51:28.709172 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702490    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702494    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702498    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702502    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702507    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702511    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702516    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702520    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702532    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702537    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702540    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702545    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702549    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702553    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702557    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702561    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702565    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702569    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702573    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702576    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:51:28.709827 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702580    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702585    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702589    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702593    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702598    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702604    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702608    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702612    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702616    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702620    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702624    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702628    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702639    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702643    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702648    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702654    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702660    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702665    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702670    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:51:28.710457 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702674    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702678    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702691    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702696    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702700    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702704    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:28.702708    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.702716    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.702921    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.705953    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.707277    2571 server.go:1019] "Starting client certificate rotation"
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.709221    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 11:51:28.711054 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.709425    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 11:51:28.713290 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.713271    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 11:51:28.716435 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.716417    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 11:51:28.724089 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.724075    2571 log.go:25] "Validated CRI v1 runtime API"
Sep 10 11:51:28.730793 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.730778    2571 log.go:25] "Validated CRI v1 image API"
Sep 10 11:51:28.731682 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.731658    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 11:51:28.739743 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.739724    2571 fs.go:135] Filesystem UUIDs: map[36614d25-acac-4907-8247-6edc49741666:/dev/nvme0n1p4 5f9fe9a5-70ae-4687-91bc-5ec3d7b7d506:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 10 11:51:28.739818 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.739742    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 11:51:28.743235 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.743217    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 11:51:28.745603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.745495    2571 manager.go:216] Machine: {Timestamp:2026-09-10 11:51:28.744151478 +0000 UTC m=+0.384722551 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100900 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec270b8142f5e0fc29a1e7f802e53403 SystemUUID:ec270b81-42f5-e0fc-29a1-e7f802e53403 BootID:404d7c4b-4abc-40bd-b524-1083d149fa47 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fe:14:54:5d:d9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fe:14:54:5d:d9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:09:36:be:12:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 11:51:28.745603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.745602    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 11:51:28.745717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.745689    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 11:51:28.748082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.748051    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 11:51:28.748227 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.748085    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-41.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 11:51:28.748800 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.748791    2571 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 11:51:28.748833 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.748803    2571 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 11:51:28.748833 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.748811    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 11:51:28.748833 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.748825    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 11:51:28.750419 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.750409    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 11:51:28.754646 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754628    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 11:51:28.754776 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754766    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 11:51:28.754807 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754797    2571 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 11:51:28.754807 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754806    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 11:51:28.754874 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754821    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 11:51:28.754874 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754834    2571 kubelet.go:398] "Adding apiserver pod source"
Sep 10 11:51:28.754874 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.754848    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 11:51:28.755499 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.755483    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 11:51:28.755557 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.755508    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 11:51:28.759452 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.758323    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 11:51:28.765413 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.765397    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 11:51:28.765481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.765427    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 11:51:28.765616 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.765596    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 11:51:28.765719 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.765624    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 11:51:28.765719 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.765636    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 11:51:28.766247 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766228    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 11:51:28.766247 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766246    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 11:51:28.766354 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766254    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 11:51:28.766916 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766905    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 11:51:28.766916 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766915    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 11:51:28.767037 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766923    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 11:51:28.767037 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766930    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 11:51:28.767037 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766939    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 11:51:28.767037 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.766947    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 11:51:28.767897 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.767886    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 11:51:28.767897 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.767896    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 11:51:28.768821 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.768796    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 11:51:28.768821 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.768796    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 11:51:28.772137 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.772124    2571 server.go:1268] "Started kubelet"
Sep 10 11:51:28.772230 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.772188    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 11:51:28.772314 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.772216    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 11:51:28.772314 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.772299    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 11:51:28.772943 ip-10-0-129-41 systemd[1]: Started Kubernetes Kubelet.
Sep 10 11:51:28.773316 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.773293    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 11:51:28.773579 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.773566    2571 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 11:51:28.778815 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.778792    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 11:51:28.781531 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.781513    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 11:51:28.782146 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782129    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 11:51:28.782258 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.782224    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:28.782781 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782754    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-41.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 11:51:28.782781 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782776    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 11:51:28.782986 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782778    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 11:51:28.782986 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782801    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 11:51:28.782986 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782932    2571 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 11:51:28.782986 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.782940    2571 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 11:51:28.783138 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783007    2571 factory.go:55] Registering systemd factory
Sep 10 11:51:28.783138 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783033    2571 factory.go:223] Registration of the systemd container factory successfully
Sep 10 11:51:28.783262 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783238    2571 factory.go:153] Registering CRI-O factory
Sep 10 11:51:28.783262 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783254    2571 factory.go:223] Registration of the crio container factory successfully
Sep 10 11:51:28.783357 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783312    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 11:51:28.783357 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783329    2571 factory.go:103] Registering Raw factory
Sep 10 11:51:28.783357 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783340    2571 manager.go:1196] Started watching for new ooms in manager
Sep 10 11:51:28.783686 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.783670    2571 manager.go:349] Starting recovery of all containers
Sep 10 11:51:28.783754 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.782685    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-41.ec2.internal.18d3f36438d6c122  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-41.ec2.internal,UID:ip-10-0-129-41.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-41.ec2.internal,},FirstTimestamp:2026-09-10 11:51:28.77209629 +0000 UTC m=+0.412667362,LastTimestamp:2026-09-10 11:51:28.77209629 +0000 UTC m=+0.412667362,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-41.ec2.internal,}"
Sep 10 11:51:28.791038 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.790996    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 11:51:28.791038 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.791027    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 11:51:28.793713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.793546    2571 manager.go:354] Recovery completed
Sep 10 11:51:28.796715 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.796670    2571 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 10 11:51:28.797183 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.797161    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fmj6k"
Sep 10 11:51:28.799888 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.799873    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.801899 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.801884    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.801957 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.801915    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.801957 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.801929    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.802466 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.802452    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 11:51:28.802466 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.802465    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 11:51:28.802585 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.802483    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 11:51:28.802585 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.802512    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.803194 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.803180    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.803271 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.803209    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.803271 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.803223    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.804028 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.803949    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-41.ec2.internal.18d3f3643a9d8186  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-41.ec2.internal,UID:ip-10-0-129-41.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-41.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-41.ec2.internal,},FirstTimestamp:2026-09-10 11:51:28.801898886 +0000 UTC m=+0.442469961,LastTimestamp:2026-09-10 11:51:28.801898886 +0000 UTC m=+0.442469961,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-41.ec2.internal,}"
Sep 10 11:51:28.804686 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.804675    2571 policy_none.go:49] "None policy: Start"
Sep 10 11:51:28.804727 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.804691    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 11:51:28.804727 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.804703    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 11:51:28.805501 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.805479    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fmj6k"
Sep 10 11:51:28.806720 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.806702    2571 policy_none.go:47] "Start"
Sep 10 11:51:28.847052 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.847021    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.847467    2571 manager.go:337] "Starting Device Plugin manager"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.847502    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.847515    2571 server.go:87] "Starting device plugin registration server"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.847804    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.847815    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.847989    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.848088    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.848096    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.848550    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.848599    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.848599    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.848625    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.848649    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.848703    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 11:51:28.859513 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.850750    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:51:28.948278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.948182    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.949240 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.949218    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"]
Sep 10 11:51:28.949312 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.949301    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.949782 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.949765    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.949851 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.949793    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.949851 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.949808    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.949851 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.949836    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.950085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.950070    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.950135 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.950092    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.950135 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.950102    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.952287 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.952274    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.952440 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.952425    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.952473 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.952461    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.953001 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.952988    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.953059 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.953008    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.953059 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.953017    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.953120 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.953060    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.953120 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.953084    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.953120 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.953101    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.955134 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.955119    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.955214 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.955143    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:51:28.955896 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.955880    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:51:28.955978 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.955908    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:51:28.955978 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.955922    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:51:28.959634 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.959619    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.959706 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.959639    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-41.ec2.internal\": node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:28.982434 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.982413    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-41.ec2.internal\" not found" node="ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.984148 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.984132    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d932f9e26f10a167bc03349232bdfffa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal\" (UID: \"d932f9e26f10a167bc03349232bdfffa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.984200 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.984155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d932f9e26f10a167bc03349232bdfffa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal\" (UID: \"d932f9e26f10a167bc03349232bdfffa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.984200 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:28.984179    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bac53f559f1c749c3a586b3ce98b441-config\") pod \"kube-apiserver-proxy-ip-10-0-129-41.ec2.internal\" (UID: \"2bac53f559f1c749c3a586b3ce98b441\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.986581 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.986565    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-41.ec2.internal\" not found" node="ip-10-0-129-41.ec2.internal"
Sep 10 11:51:28.989030 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:28.989010    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:29.084400 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.084373    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d932f9e26f10a167bc03349232bdfffa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal\" (UID: \"d932f9e26f10a167bc03349232bdfffa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.084539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.084406    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d932f9e26f10a167bc03349232bdfffa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal\" (UID: \"d932f9e26f10a167bc03349232bdfffa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.084539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.084423    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bac53f559f1c749c3a586b3ce98b441-config\") pod \"kube-apiserver-proxy-ip-10-0-129-41.ec2.internal\" (UID: \"2bac53f559f1c749c3a586b3ce98b441\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.084539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.084464    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d932f9e26f10a167bc03349232bdfffa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal\" (UID: \"d932f9e26f10a167bc03349232bdfffa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.084539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.084497    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2bac53f559f1c749c3a586b3ce98b441-config\") pod \"kube-apiserver-proxy-ip-10-0-129-41.ec2.internal\" (UID: \"2bac53f559f1c749c3a586b3ce98b441\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.084539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.084507    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d932f9e26f10a167bc03349232bdfffa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal\" (UID: \"d932f9e26f10a167bc03349232bdfffa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.089495 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:29.089481    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:29.190437 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:29.190399    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:29.285099 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.285033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.290028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:29.290008    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:29.291069 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:29.291050    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:29.391436 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:29.391406    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:30.529280 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:30.529253    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:30.629927 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:30.629858    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:30.730426 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:30.730401    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:30.744553 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.744532    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 11:51:30.744690 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.744674    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 11:51:30.831253 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:30.831228    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-41.ec2.internal\" not found"
Sep 10 11:51:30.845306 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.845277    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 11:46:29 +0000 UTC" deadline="2028-02-11 08:35:24.959909254 +0000 UTC"
Sep 10 11:51:30.845306 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.845302    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12452h43m54.114610137s"
Sep 10 11:51:30.894206 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.894102    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:51:30.912843 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:30.912810    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bac53f559f1c749c3a586b3ce98b441.slice/crio-b1a4020c2d0aa407de2c5517a636a410197cd165a0862ea2c1914e6201fc78a7 WatchSource:0}: Error finding container b1a4020c2d0aa407de2c5517a636a410197cd165a0862ea2c1914e6201fc78a7: Status 404 returned error can't find the container with id b1a4020c2d0aa407de2c5517a636a410197cd165a0862ea2c1914e6201fc78a7
Sep 10 11:51:30.913037 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:30.913022    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd932f9e26f10a167bc03349232bdfffa.slice/crio-6ed12c8476a96fd14538fa8c7654a033ecbfb49a5736991ffdd229023eb3b692 WatchSource:0}: Error finding container 6ed12c8476a96fd14538fa8c7654a033ecbfb49a5736991ffdd229023eb3b692: Status 404 returned error can't find the container with id 6ed12c8476a96fd14538fa8c7654a033ecbfb49a5736991ffdd229023eb3b692
Sep 10 11:51:30.917837 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.917823    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:51:30.919888 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.919873    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:30.935546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.935530    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 11:51:30.937406 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.937394    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal"
Sep 10 11:51:30.947185 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:30.947163    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 11:51:31.071241 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.071215    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:51:31.292691 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.292621    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:51:31.574551 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.574478    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:51:31.793362 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.793333    2571 apiserver.go:52] "Watching apiserver"
Sep 10 11:51:31.800880 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.800854    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 11:51:31.802143 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.802117    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nkxp2","kube-system/konnectivity-agent-2lxrm","openshift-dns/node-resolver-v4ghr","openshift-image-registry/node-ca-dxl2z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal","openshift-multus/multus-additional-cni-plugins-lt6jn","openshift-multus/multus-jcf6l","openshift-multus/network-metrics-daemon-cfwkr","kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92","openshift-cluster-node-tuning-operator/tuned-7jssl","openshift-network-diagnostics/network-check-target-4j9sk","openshift-network-operator/iptables-alerter-lln46","openshift-ovn-kubernetes/ovnkube-node-9p6c4"]
Sep 10 11:51:31.804634 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.804613    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.804734 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.804695    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:31.806712 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.806689    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.808840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.808819    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.811540 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.811102    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.813465 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.813257    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.816168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.816124    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.818082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.818058    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9q5nc\""
Sep 10 11:51:31.818465 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.818444    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:31.818557 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.818515    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:31.819278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.819253    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 11:51:31.819377 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.819358    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 11:51:31.820449 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.820431    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.821793 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.821774    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.822042 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822023    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.822259 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822243    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zgznx\""
Sep 10 11:51:31.822628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822609    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 11:51:31.822700 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822664    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 11:51:31.822769 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822699    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6jz4s\""
Sep 10 11:51:31.822818 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822808    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 11:51:31.822871 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822814    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 11:51:31.822923 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.822873    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.823072 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.823055    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.823189 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.823172    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 11:51:31.823300 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.823285    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.823409 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.823391    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 11:51:31.823461 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.823453    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.823552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.823515    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-q79lk\""
Sep 10 11:51:31.825553 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.825417    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:31.825553 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.825476    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:31.827535 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.827516    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.827853 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.827837    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:31.830309 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-socket-dir-parent\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830381 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830323    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-k8s-cni-cncf-io\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830381 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830347    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-cni-bin\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830381 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830372    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-cni-multus\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830397    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-etc-kubernetes\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830422    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-system-cni-dir\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.830476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830444    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-cnibin\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830465    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e193539-5ca1-4471-b03a-edba883854ad-agent-certs\") pod \"konnectivity-agent-2lxrm\" (UID: \"1e193539-5ca1-4471-b03a-edba883854ad\") " pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.830603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.830603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830517    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8gw\" (UniqueName: \"kubernetes.io/projected/89662635-b5c4-47b7-8e37-b30f4b58fda4-kube-api-access-cg8gw\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.830603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b8668d6-db4b-43c5-8567-51fcb8429334-serviceca\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.830603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830560    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.830603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-multus-certs\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830603 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830600    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-cnibin\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830629    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-registration-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82clh\" (UniqueName: \"kubernetes.io/projected/61c2ef99-6212-41d2-aac3-2c6ef1f04338-kube-api-access-82clh\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830719    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92c58a92-66b4-4673-887e-b84c82a92d80-kubelet-config\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830818    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b8668d6-db4b-43c5-8567-51fcb8429334-host\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf25d\" (UniqueName: \"kubernetes.io/projected/775b1046-09eb-4c3e-b077-1f6b7102ee99-kube-api-access-hf25d\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.830887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e193539-5ca1-4471-b03a-edba883854ad-konnectivity-ca\") pod \"konnectivity-agent-2lxrm\" (UID: \"1e193539-5ca1-4471-b03a-edba883854ad\") " pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830889    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-sys-fs\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830912    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830946    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-kubelet\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830969    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-hostroot\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.830992    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92c58a92-66b4-4673-887e-b84c82a92d80-dbus\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831014    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89662635-b5c4-47b7-8e37-b30f4b58fda4-tmp-dir\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-cni-binary-copy\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831103    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/775b1046-09eb-4c3e-b077-1f6b7102ee99-cni-binary-copy\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831145    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-conf-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:31.831233 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831226    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-etc-selinux\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831256    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-cni-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831278    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-netns\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-daemon-config\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg9ft\" (UniqueName: \"kubernetes.io/projected/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-kube-api-access-lg9ft\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831355    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9cgx\" (UniqueName: \"kubernetes.io/projected/0b8668d6-db4b-43c5-8567-51fcb8429334-kube-api-access-w9cgx\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831473    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nchzw\" (UniqueName: \"kubernetes.io/projected/c50f797b-4d46-458f-b5b0-f1d143a40e92-kube-api-access-nchzw\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-system-cni-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/89662635-b5c4-47b7-8e37-b30f4b58fda4-hosts-file\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-socket-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.831713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.831667    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-device-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.835576 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.834564    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.835576 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.834742    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.836850 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.836828    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h8gnj\""
Sep 10 11:51:31.837094 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.837075    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j9fb5\""
Sep 10 11:51:31.837388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.837371    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 11:51:31.837604 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.837591    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.838335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.838317    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87ssd\""
Sep 10 11:51:31.838859 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.838840    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 11:51:31.839238 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.839221    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.839355 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.839333    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.839717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.839470    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p8hlq\""
Sep 10 11:51:31.839830 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.839775    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 11:51:31.842755 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.842736    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 11:51:31.842927 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.842906    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.842997 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.842924    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 11:51:31.843057 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.842751    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 11:51:31.843148 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.842824    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 11:51:31.843346 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.843322    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 11:51:31.843436 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.843136    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9mhmm\""
Sep 10 11:51:31.847369 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.847322    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 11:51:31.890671 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.890627    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal" event={"ID":"d932f9e26f10a167bc03349232bdfffa","Type":"ContainerStarted","Data":"6ed12c8476a96fd14538fa8c7654a033ecbfb49a5736991ffdd229023eb3b692"}
Sep 10 11:51:31.892955 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.892933    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal" event={"ID":"2bac53f559f1c749c3a586b3ce98b441","Type":"ContainerStarted","Data":"b1a4020c2d0aa407de2c5517a636a410197cd165a0862ea2c1914e6201fc78a7"}
Sep 10 11:51:31.921284 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.921263    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 11:51:31.932781 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b8668d6-db4b-43c5-8567-51fcb8429334-serviceca\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.932891 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932790    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.932891 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932815    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.932891 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932839    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82clh\" (UniqueName: \"kubernetes.io/projected/61c2ef99-6212-41d2-aac3-2c6ef1f04338-kube-api-access-82clh\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.932891 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932855    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92c58a92-66b4-4673-887e-b84c82a92d80-kubelet-config\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.932891 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932871    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b8668d6-db4b-43c5-8567-51fcb8429334-host\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932892    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-env-overrides\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932937    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-hostroot\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932952    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-log-socket\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932978    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-var-lib-kubelet\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.932993    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89662635-b5c4-47b7-8e37-b30f4b58fda4-tmp-dir\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933007    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-conf-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933021    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933036    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-systemd\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovnkube-script-lib\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933082    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-etc-selinux\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-run-netns\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933139    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-kubernetes\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933176    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-k8s-cni-cncf-io\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933195    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-cni-bin\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933234    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/89662635-b5c4-47b7-8e37-b30f4b58fda4-hosts-file\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933253    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-cni-netd\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933272    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysconfig\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933286    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-device-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933310    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933311    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0b8668d6-db4b-43c5-8567-51fcb8429334-serviceca\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933324    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-modprobe-d\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933348    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-system-cni-dir\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933372    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e193539-5ca1-4471-b03a-edba883854ad-agent-certs\") pod \"konnectivity-agent-2lxrm\" (UID: \"1e193539-5ca1-4471-b03a-edba883854ad\") " pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.933840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933790    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933888    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-var-lib-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933916    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovn-node-metrics-cert\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933942    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1589c2b5-976a-4afe-88a7-86166a8d63a5-tmp\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.933972    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8gw\" (UniqueName: \"kubernetes.io/projected/89662635-b5c4-47b7-8e37-b30f4b58fda4-kube-api-access-cg8gw\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934000    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-multus-certs\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934026    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-etc-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934055    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqqph\" (UniqueName: \"kubernetes.io/projected/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-kube-api-access-cqqph\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934082    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-cnibin\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-registration-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934137    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf25d\" (UniqueName: \"kubernetes.io/projected/775b1046-09eb-4c3e-b077-1f6b7102ee99-kube-api-access-hf25d\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e193539-5ca1-4471-b03a-edba883854ad-konnectivity-ca\") pod \"konnectivity-agent-2lxrm\" (UID: \"1e193539-5ca1-4471-b03a-edba883854ad\") " pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934206    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-systemd\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-host-slash\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934257    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-sys-fs\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934284    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-kubelet\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.934481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934306    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-sys\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-iptables-alerter-script\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92c58a92-66b4-4673-887e-b84c82a92d80-dbus\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-cni-binary-copy\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934412    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/775b1046-09eb-4c3e-b077-1f6b7102ee99-cni-binary-copy\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934438    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934465    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-host\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934490    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-cni-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-netns\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-daemon-config\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg9ft\" (UniqueName: \"kubernetes.io/projected/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-kube-api-access-lg9ft\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-cni-bin\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934621    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovnkube-config\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934645    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysctl-conf\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934670    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-socket-dir-parent\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-kubelet\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934720    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-systemd-units\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.935296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9cgx\" (UniqueName: \"kubernetes.io/projected/0b8668d6-db4b-43c5-8567-51fcb8429334-kube-api-access-w9cgx\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934786    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nchzw\" (UniqueName: \"kubernetes.io/projected/c50f797b-4d46-458f-b5b0-f1d143a40e92-kube-api-access-nchzw\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934813    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-system-cni-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934844    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-socket-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934869    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934898    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-cni-multus\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934923    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-etc-kubernetes\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934951    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-ovn\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.934980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysctl-d\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-run\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935038    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-lib-modules\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-tuned\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935084    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxdmn\" (UniqueName: \"kubernetes.io/projected/1589c2b5-976a-4afe-88a7-86166a8d63a5-kube-api-access-wxdmn\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-cnibin\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935143    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-slash\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935189    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-node-log\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935214    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkxng\" (UniqueName: \"kubernetes.io/projected/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-kube-api-access-hkxng\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:31.936097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935242    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935329    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935620    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/92c58a92-66b4-4673-887e-b84c82a92d80-kubelet-config\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.935669    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b8668d6-db4b-43c5-8567-51fcb8429334-host\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.936127    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.936210    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-hostroot\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.936480    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89662635-b5c4-47b7-8e37-b30f4b58fda4-tmp-dir\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.936544    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-conf-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.936631    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:31.936949 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.936689    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:32.436669779 +0000 UTC m=+3.039944682 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:31.937413 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937082    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-multus-certs\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.937413 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937180    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-cnibin\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.937413 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937241    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-socket-dir-parent\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.937413 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937287    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-registration-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937492    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1e193539-5ca1-4471-b03a-edba883854ad-agent-certs\") pod \"konnectivity-agent-2lxrm\" (UID: \"1e193539-5ca1-4471-b03a-edba883854ad\") " pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937540    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-etc-kubernetes\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.937557    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937621    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-kubelet\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937626    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-cni-multus\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937635    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-etc-selinux\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937677    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-system-cni-dir\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:31.937693    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:32.437675822 +0000 UTC m=+3.040950736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937690    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-cnibin\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937718    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-var-lib-cni-bin\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937764    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-system-cni-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937769    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c50f797b-4d46-458f-b5b0-f1d143a40e92-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937755    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/92c58a92-66b4-4673-887e-b84c82a92d80-dbus\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937807    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-device-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937763    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-k8s-cni-cncf-io\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937857    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-sys-fs\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.938170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937862    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/89662635-b5c4-47b7-8e37-b30f4b58fda4-hosts-file\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.939173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937914    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-host-run-netns\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.939173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937926    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1e193539-5ca1-4471-b03a-edba883854ad-konnectivity-ca\") pod \"konnectivity-agent-2lxrm\" (UID: \"1e193539-5ca1-4471-b03a-edba883854ad\") " pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:31.939173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.937999    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-cni-dir\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.939173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.938108    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c50f797b-4d46-458f-b5b0-f1d143a40e92-cni-binary-copy\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:31.939173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.938541    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61c2ef99-6212-41d2-aac3-2c6ef1f04338-socket-dir\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.939173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.939129    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/775b1046-09eb-4c3e-b077-1f6b7102ee99-cni-binary-copy\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.939468 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.939174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/775b1046-09eb-4c3e-b077-1f6b7102ee99-multus-daemon-config\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.944418 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.944397    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw74x"
Sep 10 11:51:31.951425 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.951404    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82clh\" (UniqueName: \"kubernetes.io/projected/61c2ef99-6212-41d2-aac3-2c6ef1f04338-kube-api-access-82clh\") pod \"aws-ebs-csi-driver-node-r2h92\" (UID: \"61c2ef99-6212-41d2-aac3-2c6ef1f04338\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:31.953588 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.953570    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf25d\" (UniqueName: \"kubernetes.io/projected/775b1046-09eb-4c3e-b077-1f6b7102ee99-kube-api-access-hf25d\") pod \"multus-jcf6l\" (UID: \"775b1046-09eb-4c3e-b077-1f6b7102ee99\") " pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:31.958655 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.958632    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw74x"
Sep 10 11:51:31.959280 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.959175    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9cgx\" (UniqueName: \"kubernetes.io/projected/0b8668d6-db4b-43c5-8567-51fcb8429334-kube-api-access-w9cgx\") pod \"node-ca-dxl2z\" (UID: \"0b8668d6-db4b-43c5-8567-51fcb8429334\") " pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:31.959374 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.959353    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8gw\" (UniqueName: \"kubernetes.io/projected/89662635-b5c4-47b7-8e37-b30f4b58fda4-kube-api-access-cg8gw\") pod \"node-resolver-v4ghr\" (UID: \"89662635-b5c4-47b7-8e37-b30f4b58fda4\") " pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:31.960286 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.960265    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg9ft\" (UniqueName: \"kubernetes.io/projected/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-kube-api-access-lg9ft\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:31.962389 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:31.962368    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nchzw\" (UniqueName: \"kubernetes.io/projected/c50f797b-4d46-458f-b5b0-f1d143a40e92-kube-api-access-nchzw\") pod \"multus-additional-cni-plugins-lt6jn\" (UID: \"c50f797b-4d46-458f-b5b0-f1d143a40e92\") " pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:32.035755 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035712    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.035755 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-host\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.035998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035789    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-cni-bin\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.035998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035812    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovnkube-config\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.035998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035822    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.035998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035890    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-host\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.035998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035940    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-cni-bin\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.035998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysctl-conf\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.035834    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysctl-conf\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036035    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-kubelet\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036061    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-systemd-units\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-ovn\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036129    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysctl-d\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036168    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-run\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036190    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-lib-modules\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-tuned\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxdmn\" (UniqueName: \"kubernetes.io/projected/1589c2b5-976a-4afe-88a7-86166a8d63a5-kube-api-access-wxdmn\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036261    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-slash\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036285    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-node-log\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036309    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkxng\" (UniqueName: \"kubernetes.io/projected/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-kube-api-access-hkxng\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036362    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-env-overrides\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-log-socket\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036411    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-var-lib-kubelet\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036440    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-lib-modules\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036492    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-systemd\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036509    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovnkube-config\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-kubelet\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036568    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-systemd-units\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036602    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-ovn\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.036715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036707    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysctl-d\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-systemd\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovnkube-script-lib\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-node-log\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036793    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-run\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036816    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-run-netns\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036838    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-kubernetes\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036851    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-slash\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036864    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-cni-netd\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036886    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysconfig\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-modprobe-d\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036969    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-var-lib-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-run-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.036992    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovn-node-metrics-cert\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1589c2b5-976a-4afe-88a7-86166a8d63a5-tmp\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.037337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-log-socket\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037047    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-etc-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqqph\" (UniqueName: \"kubernetes.io/projected/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-kube-api-access-cqqph\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037085    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-var-lib-kubelet\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037099    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-systemd\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037121    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037122    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-host-slash\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037171    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-host-slash\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-sys\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-iptables-alerter-script\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037209    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-sys\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037258    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-env-overrides\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037269    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-sysconfig\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037317    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-modprobe-d\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037319    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-var-lib-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-run-netns\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037356    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-host-cni-netd\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037412    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-kubernetes\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037877    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovnkube-script-lib\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.038953 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037898    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-iptables-alerter-script\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.038953 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037963    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-systemd\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.038953 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.037988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-etc-openvswitch\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.039106 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.039045    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1589c2b5-976a-4afe-88a7-86166a8d63a5-etc-tuned\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.039693 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.039669    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-ovn-node-metrics-cert\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.040125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.040099    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1589c2b5-976a-4afe-88a7-86166a8d63a5-tmp\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.044033 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.044007    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:51:32.044033 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.044030    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:51:32.044198 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.044042    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:32.044198 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.044101    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:51:32.54408455 +0000 UTC m=+3.147359442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:32.045854 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.045821    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkxng\" (UniqueName: \"kubernetes.io/projected/522fb0c4-1e0d-4f53-a339-77d6ccad4cb8-kube-api-access-hkxng\") pod \"iptables-alerter-lln46\" (UID: \"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8\") " pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.046366 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.046343    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqqph\" (UniqueName: \"kubernetes.io/projected/0871f625-b780-4b5c-b20e-0e58dbc1c9e7-kube-api-access-cqqph\") pod \"ovnkube-node-9p6c4\" (UID: \"0871f625-b780-4b5c-b20e-0e58dbc1c9e7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.046987 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.046959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxdmn\" (UniqueName: \"kubernetes.io/projected/1589c2b5-976a-4afe-88a7-86166a8d63a5-kube-api-access-wxdmn\") pod \"tuned-7jssl\" (UID: \"1589c2b5-976a-4afe-88a7-86166a8d63a5\") " pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.119240 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.119134    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:32.127622 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.127591    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v4ghr"
Sep 10 11:51:32.144701 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.144663    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dxl2z"
Sep 10 11:51:32.149481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.149457    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lt6jn"
Sep 10 11:51:32.156003 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.155980    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jcf6l"
Sep 10 11:51:32.164002 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.163981    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92"
Sep 10 11:51:32.171772 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.171752    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7jssl"
Sep 10 11:51:32.180293 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.180268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lln46"
Sep 10 11:51:32.184889 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.184867    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:32.439908 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.439831    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:32.440085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.439918    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:32.440085 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.440002    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:32.440085 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.440036    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:32.440085 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.440077    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:33.440058211 +0000 UTC m=+4.043333118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:32.440261 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.440096    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:33.440088617 +0000 UTC m=+4.043363506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:32.641348 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.641305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:32.641527 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.641505    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:51:32.641527 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.641527    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:51:32.641643 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.641542    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:32.641643 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.641590    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:51:33.64157785 +0000 UTC m=+4.244852739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:32.886600 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.886535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:32.887028 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:32.886642    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:32.929450 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.929307    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1589c2b5_976a_4afe_88a7_86166a8d63a5.slice/crio-9759337974fa2b2a717a18f98efa16d7043be441ac087ab8f5aaf40fb21f2b30 WatchSource:0}: Error finding container 9759337974fa2b2a717a18f98efa16d7043be441ac087ab8f5aaf40fb21f2b30: Status 404 returned error can't find the container with id 9759337974fa2b2a717a18f98efa16d7043be441ac087ab8f5aaf40fb21f2b30
Sep 10 11:51:32.930103 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.930073    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522fb0c4_1e0d_4f53_a339_77d6ccad4cb8.slice/crio-96168faae1c31d201e47dfbe199dbc2c231dedddc9a517cb2a04db150ff21f26 WatchSource:0}: Error finding container 96168faae1c31d201e47dfbe199dbc2c231dedddc9a517cb2a04db150ff21f26: Status 404 returned error can't find the container with id 96168faae1c31d201e47dfbe199dbc2c231dedddc9a517cb2a04db150ff21f26
Sep 10 11:51:32.931010 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.930990    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e193539_5ca1_4471_b03a_edba883854ad.slice/crio-ac0be1739e02ef84a20a5d3bc088127a0b5462655617c222a4680d683ee5e2ac WatchSource:0}: Error finding container ac0be1739e02ef84a20a5d3bc088127a0b5462655617c222a4680d683ee5e2ac: Status 404 returned error can't find the container with id ac0be1739e02ef84a20a5d3bc088127a0b5462655617c222a4680d683ee5e2ac
Sep 10 11:51:32.931531 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.931510    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0871f625_b780_4b5c_b20e_0e58dbc1c9e7.slice/crio-1418bba2f58e55e1cfb43774854451f3e6e2f83f4a254ad5468f8a1d6ace59a3 WatchSource:0}: Error finding container 1418bba2f58e55e1cfb43774854451f3e6e2f83f4a254ad5468f8a1d6ace59a3: Status 404 returned error can't find the container with id 1418bba2f58e55e1cfb43774854451f3e6e2f83f4a254ad5468f8a1d6ace59a3
Sep 10 11:51:32.935043 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.934978    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8668d6_db4b_43c5_8567_51fcb8429334.slice/crio-afda832ddcea453131ed5da764c12afbc4c5b8a8afb0d3e30197a8c832dbc898 WatchSource:0}: Error finding container afda832ddcea453131ed5da764c12afbc4c5b8a8afb0d3e30197a8c832dbc898: Status 404 returned error can't find the container with id afda832ddcea453131ed5da764c12afbc4c5b8a8afb0d3e30197a8c832dbc898
Sep 10 11:51:32.936463 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.936423    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c2ef99_6212_41d2_aac3_2c6ef1f04338.slice/crio-a84ef723d77ed3a9221821b03280043cc0e1100a0f12d320885ac2bc62c5d9bf WatchSource:0}: Error finding container a84ef723d77ed3a9221821b03280043cc0e1100a0f12d320885ac2bc62c5d9bf: Status 404 returned error can't find the container with id a84ef723d77ed3a9221821b03280043cc0e1100a0f12d320885ac2bc62c5d9bf
Sep 10 11:51:32.937463 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.937393    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89662635_b5c4_47b7_8e37_b30f4b58fda4.slice/crio-8b0e4989d5c88a76a5f0bc5c6ffbc842aca83d0421c0dd020fc72b3d444c1674 WatchSource:0}: Error finding container 8b0e4989d5c88a76a5f0bc5c6ffbc842aca83d0421c0dd020fc72b3d444c1674: Status 404 returned error can't find the container with id 8b0e4989d5c88a76a5f0bc5c6ffbc842aca83d0421c0dd020fc72b3d444c1674
Sep 10 11:51:32.940200 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.940178    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc50f797b_4d46_458f_b5b0_f1d143a40e92.slice/crio-2f7fd195d2edbcbb403788c4a3766d1929b351c2a7941e033fa0712efd1c5bb1 WatchSource:0}: Error finding container 2f7fd195d2edbcbb403788c4a3766d1929b351c2a7941e033fa0712efd1c5bb1: Status 404 returned error can't find the container with id 2f7fd195d2edbcbb403788c4a3766d1929b351c2a7941e033fa0712efd1c5bb1
Sep 10 11:51:32.941615 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:51:32.941592    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775b1046_09eb_4c3e_b077_1f6b7102ee99.slice/crio-f48e99c25c3f717fa863db4280ac68a44ac46d8f471f63e035b6911cc019cf5a WatchSource:0}: Error finding container f48e99c25c3f717fa863db4280ac68a44ac46d8f471f63e035b6911cc019cf5a: Status 404 returned error can't find the container with id f48e99c25c3f717fa863db4280ac68a44ac46d8f471f63e035b6911cc019cf5a
Sep 10 11:51:32.962885 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.962858    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 11:46:31 +0000 UTC" deadline="2028-02-08 07:46:25.445095387 +0000 UTC"
Sep 10 11:51:32.962988 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:32.962885    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12379h54m52.482213938s"
Sep 10 11:51:33.445955 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.445873    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:33.446100 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.445977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:33.446100 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.446066    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:33.446100 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.446092    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:33.446273 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.446171    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:35.446129526 +0000 UTC m=+6.049404429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:33.446273 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.446192    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:35.446181642 +0000 UTC m=+6.049456536 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:33.647768 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.647728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:33.648024 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.648005    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:51:33.648095 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.648030    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:51:33.648095 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.648044    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:33.648232 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.648102    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:51:35.648084718 +0000 UTC m=+6.251359625 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:33.887179 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.887099    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:33.887871 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.887247    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:33.887871 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.887664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:33.887871 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:33.887739    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:33.904342 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.904308    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7jssl" event={"ID":"1589c2b5-976a-4afe-88a7-86166a8d63a5","Type":"ContainerStarted","Data":"9759337974fa2b2a717a18f98efa16d7043be441ac087ab8f5aaf40fb21f2b30"}
Sep 10 11:51:33.911826 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.911797    2571 generic.go:356] "Generic (PLEG): container finished" podID="d932f9e26f10a167bc03349232bdfffa" containerID="b7bce48459b85aee47ed2223fce71ce2eb9f1d93e51c9d5f40bf39e472e79e3c" exitCode=0
Sep 10 11:51:33.911958 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.911878    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal" event={"ID":"d932f9e26f10a167bc03349232bdfffa","Type":"ContainerDied","Data":"b7bce48459b85aee47ed2223fce71ce2eb9f1d93e51c9d5f40bf39e472e79e3c"}
Sep 10 11:51:33.935625 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.935568    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal" event={"ID":"2bac53f559f1c749c3a586b3ce98b441","Type":"ContainerStarted","Data":"4c0d5fb5e42d492474d4f695b0d43a6ea39106c2ecbdb6f4238b281618914975"}
Sep 10 11:51:33.959489 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.959420    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jcf6l" event={"ID":"775b1046-09eb-4c3e-b077-1f6b7102ee99","Type":"ContainerStarted","Data":"f48e99c25c3f717fa863db4280ac68a44ac46d8f471f63e035b6911cc019cf5a"}
Sep 10 11:51:33.964026 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.963982    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 11:46:31 +0000 UTC" deadline="2028-04-09 20:13:43.899420882 +0000 UTC"
Sep 10 11:51:33.964026 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.964027    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13856h22m9.93539741s"
Sep 10 11:51:33.965534 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.965507    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92" event={"ID":"61c2ef99-6212-41d2-aac3-2c6ef1f04338","Type":"ContainerStarted","Data":"a84ef723d77ed3a9221821b03280043cc0e1100a0f12d320885ac2bc62c5d9bf"}
Sep 10 11:51:33.975601 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.975575    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dxl2z" event={"ID":"0b8668d6-db4b-43c5-8567-51fcb8429334","Type":"ContainerStarted","Data":"afda832ddcea453131ed5da764c12afbc4c5b8a8afb0d3e30197a8c832dbc898"}
Sep 10 11:51:33.984070 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.984045    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v4ghr" event={"ID":"89662635-b5c4-47b7-8e37-b30f4b58fda4","Type":"ContainerStarted","Data":"8b0e4989d5c88a76a5f0bc5c6ffbc842aca83d0421c0dd020fc72b3d444c1674"}
Sep 10 11:51:33.987839 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.987812    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerStarted","Data":"2f7fd195d2edbcbb403788c4a3766d1929b351c2a7941e033fa0712efd1c5bb1"}
Sep 10 11:51:34.000542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.996058    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lln46" event={"ID":"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8","Type":"ContainerStarted","Data":"96168faae1c31d201e47dfbe199dbc2c231dedddc9a517cb2a04db150ff21f26"}
Sep 10 11:51:34.000542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:33.997164    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"1418bba2f58e55e1cfb43774854451f3e6e2f83f4a254ad5468f8a1d6ace59a3"}
Sep 10 11:51:34.000542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:34.000340    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2lxrm" event={"ID":"1e193539-5ca1-4471-b03a-edba883854ad","Type":"ContainerStarted","Data":"ac0be1739e02ef84a20a5d3bc088127a0b5462655617c222a4680d683ee5e2ac"}
Sep 10 11:51:34.886931 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:34.886860    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:34.887099 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:34.886998    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:35.020472 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.020438    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal" event={"ID":"d932f9e26f10a167bc03349232bdfffa","Type":"ContainerStarted","Data":"f8f17439430852cff775bb2e3e8a28ace02c6578543eaf768fe7f384a2aa4ded"}
Sep 10 11:51:35.037526 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.037474    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-41.ec2.internal" podStartSLOduration=5.037454691 podStartE2EDuration="5.037454691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:35.037345695 +0000 UTC m=+5.640620607" watchObservedRunningTime="2026-09-10 11:51:35.037454691 +0000 UTC m=+5.640729603"
Sep 10 11:51:35.037844 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.037814    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-41.ec2.internal" podStartSLOduration=5.037806394 podStartE2EDuration="5.037806394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:33.955399816 +0000 UTC m=+4.558674730" watchObservedRunningTime="2026-09-10 11:51:35.037806394 +0000 UTC m=+5.641081305"
Sep 10 11:51:35.463760 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.463720    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:35.463923 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.463787    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:35.463990 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.463922    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:35.464054 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.463990    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:39.463973675 +0000 UTC m=+10.067248570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:35.464447 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.464396    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:35.464528 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.464451    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:39.464435366 +0000 UTC m=+10.067710270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:35.664809 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.664772    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:35.664968 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.664944    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:51:35.664968 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.664966    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:51:35.665091 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.664979    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:35.665091 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.665036    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:51:39.665017461 +0000 UTC m=+10.268292369 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:35.893340 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.893268    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:35.893497 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.893383    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:35.893497 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:35.893445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:35.893610 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:35.893506    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:36.886906 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:36.886871    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:36.887377 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:36.887022    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:37.890605 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:37.890574    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:37.891029 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:37.890698    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:37.891029 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:37.890973    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:37.891174 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:37.891042    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:38.887329 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:38.886836    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:38.887329 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:38.886974    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:39.498117 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:39.498083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:39.498573 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:39.498179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:39.498573 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.498284    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:39.498573 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.498286    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:39.498573 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.498341    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:47.498322214 +0000 UTC m=+18.101597108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:39.498573 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.498359    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:51:47.498349358 +0000 UTC m=+18.101624250 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:39.700118 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:39.699592    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:39.700118 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.699802    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:51:39.700118 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.699821    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:51:39.700118 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.699835    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:39.700118 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.699932    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:51:47.699877642 +0000 UTC m=+18.303152532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:39.890784 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:39.890681    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:39.890937 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.890790    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:39.890995 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:39.890966    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:39.891146 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:39.891121    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:40.886953 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:40.886908    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:40.887348 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:40.887077    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:41.886698 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:41.886664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:41.886698 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:41.886702    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:41.886935 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:41.886786    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:41.886935 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:41.886901    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:42.886316 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:42.886288    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:42.886490 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:42.886404    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:43.886145 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:43.886114    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:43.886558 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:43.886131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:43.886558 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:43.886265    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:43.886558 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:43.886326    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:44.887125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:44.887091    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:44.887588 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:44.887249    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:45.886699 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:45.886657    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:45.886891 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:45.886666    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:45.886891 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:45.886810    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:45.886891 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:45.886860    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:46.886930 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:46.886896    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:46.887386 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:46.887028    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:47.553702 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:47.553670    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:47.553889 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:47.553720    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:47.553889 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.553834    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:47.553889 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.553837    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:47.554052 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.553894    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:03.553877169 +0000 UTC m=+34.157152075 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:51:47.554052 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.553912    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:03.553903309 +0000 UTC m=+34.157178198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:51:47.755113 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:47.755076    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:47.755280 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.755248    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:51:47.755280 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.755271    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:51:47.755384 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.755283    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:47.755384 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.755340    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:52:03.755323317 +0000 UTC m=+34.358598223 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:51:47.886800 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:47.886724    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:47.886945 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:47.886724    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:47.886945 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.886861    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:47.887395 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:47.886962    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:48.886611 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:48.886579    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:48.886788 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:48.886731    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:49.887646 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:49.887609    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:49.888097 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:49.887728    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:49.888097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:49.887756    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:49.888097 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:49.887849    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:50.887028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:50.886868    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:50.887122 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:50.887105    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:51.061130 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.061072    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v4ghr" event={"ID":"89662635-b5c4-47b7-8e37-b30f4b58fda4","Type":"ContainerStarted","Data":"2c6ca34431d1382079ab7c0c59a3213e0161d10fb40b36af23cb49466ee405f7"}
Sep 10 11:51:51.062630 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.062610    2571 generic.go:356] "Generic (PLEG): container finished" podID="c50f797b-4d46-458f-b5b0-f1d143a40e92" containerID="270c654e92d763249dee8e5a1ddf6c33cf5c194ccc4427b863bb2f477491b5d6" exitCode=0
Sep 10 11:51:51.062705 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.062677    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerDied","Data":"270c654e92d763249dee8e5a1ddf6c33cf5c194ccc4427b863bb2f477491b5d6"}
Sep 10 11:51:51.068644 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.068619    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"ca83c022fe5c2529108e61f9359da1fc1259a2df88cd8a975285ce694d85c3a4"}
Sep 10 11:51:51.068751 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.068725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"6e4df9cfd0884a684e574b9548b3cee76470ef51e1a882f28d1ac4ae497b6a97"}
Sep 10 11:51:51.068806 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.068788    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"9a6a34bb92263ab6077cfc7e22458eff2c493c8c2b23919468f6ab9d3e1dd53c"}
Sep 10 11:51:51.068852 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.068836    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"d018679c94fdf7387b28d5a038cef360955a54bc9d3ed10f4295b3172d2da51e"}
Sep 10 11:51:51.068893 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.068880    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"649e0e05089c0de1fe3a62a33dd92551702b0fef95b65af85a518f9b495b3619"}
Sep 10 11:51:51.070356 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.070329    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2lxrm" event={"ID":"1e193539-5ca1-4471-b03a-edba883854ad","Type":"ContainerStarted","Data":"8afb3729a6810f6a3b46abfda41064d3b10eb326e7e23dc1aaa5e200fecc7f1d"}
Sep 10 11:51:51.072027 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.072005    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7jssl" event={"ID":"1589c2b5-976a-4afe-88a7-86166a8d63a5","Type":"ContainerStarted","Data":"b31063903c4af6197452af363a144e38ee6269379d4450c65b4e437fb329f762"}
Sep 10 11:51:51.073464 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.073443    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jcf6l" event={"ID":"775b1046-09eb-4c3e-b077-1f6b7102ee99","Type":"ContainerStarted","Data":"642ad704b81ec5bbb6324188a8ffe39552455e6340097dc23256c4e2c2abac00"}
Sep 10 11:51:51.074816 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.074792    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92" event={"ID":"61c2ef99-6212-41d2-aac3-2c6ef1f04338","Type":"ContainerStarted","Data":"385ccb720f2c0d2db6e3a868e2f5e031eba4b625a4f7cb96fabcbb219ee8b525"}
Sep 10 11:51:51.076135 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.076116    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dxl2z" event={"ID":"0b8668d6-db4b-43c5-8567-51fcb8429334","Type":"ContainerStarted","Data":"68acdcf6d2f30b5d07660b3da0fc22bda6bb88cdd0f72e2521b6731aa5a3308a"}
Sep 10 11:51:51.080105 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.080068    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v4ghr" podStartSLOduration=3.596426621 podStartE2EDuration="21.080057074s" totalImagesPullingTime="17.483630453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:51.079700389 +0000 UTC m=+21.682975300" watchObservedRunningTime="2026-09-10 11:51:51.080057074 +0000 UTC m=+21.683331985"
Sep 10 11:51:51.099554 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.099509    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7jssl" podStartSLOduration=3.606716673 podStartE2EDuration="21.09949906s" totalImagesPullingTime="17.492782387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:51.099369002 +0000 UTC m=+21.702644130" watchObservedRunningTime="2026-09-10 11:51:51.09949906 +0000 UTC m=+21.702773971"
Sep 10 11:51:51.155505 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.155465    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2lxrm" podStartSLOduration=3.665021926 podStartE2EDuration="21.155452543s" totalImagesPullingTime="17.490430617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:51.155038046 +0000 UTC m=+21.758312958" watchObservedRunningTime="2026-09-10 11:51:51.155452543 +0000 UTC m=+21.758727453"
Sep 10 11:51:51.190160 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.190123    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dxl2z" podStartSLOduration=7.424191281 podStartE2EDuration="21.190111058s" totalImagesPullingTime="13.765919777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:51.189862552 +0000 UTC m=+21.793137464" watchObservedRunningTime="2026-09-10 11:51:51.190111058 +0000 UTC m=+21.793385966"
Sep 10 11:51:51.216947 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.216909    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jcf6l" podStartSLOduration=3.7164198649999998 podStartE2EDuration="21.216897553s" totalImagesPullingTime="17.500477688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:51.216842374 +0000 UTC m=+21.820117285" watchObservedRunningTime="2026-09-10 11:51:51.216897553 +0000 UTC m=+21.820172464"
Sep 10 11:51:51.340595 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.340516    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:51.341133 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.341105    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:51.886571 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.886434    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:51.886657 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.886499    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:51.886657 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:51.886648    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:51.886736 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:51.886723    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:51.898334 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:51.898314    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 11:51:52.080736 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.080633    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lln46" event={"ID":"522fb0c4-1e0d-4f53-a339-77d6ccad4cb8","Type":"ContainerStarted","Data":"aadac785874bfd3babc9ad19ff1f2a58ec582935e9a6562ed833b400676c4e72"}
Sep 10 11:51:52.084274 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.084233    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"7ecd68347931014685cbd6a50849bbe60fd54ca56f5c38c792d8ed6f894b813f"}
Sep 10 11:51:52.086079 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.086054    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92" event={"ID":"61c2ef99-6212-41d2-aac3-2c6ef1f04338","Type":"ContainerStarted","Data":"0c0d0fa68afd7d563f0f9bfda7f37eec9fc0c07c7d37aa4f164b388565e7074e"}
Sep 10 11:51:52.086852 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.086786    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:52.087375 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.087359    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2lxrm"
Sep 10 11:51:52.099096 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.099053    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lln46" podStartSLOduration=4.622017575 podStartE2EDuration="22.099041363s" totalImagesPullingTime="17.477023788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:52.098639141 +0000 UTC m=+22.701914053" watchObservedRunningTime="2026-09-10 11:51:52.099041363 +0000 UTC m=+22.702316274"
Sep 10 11:51:52.887133 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.887098    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:52.887336 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:52.887265    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:52.896902 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.896802    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T11:51:51.898330155Z","UUID":"6064c622-dd01-4679-b965-d4f4cd0b5d22","Handler":null,"Name":"","Endpoint":""}
Sep 10 11:51:52.898689 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.898662    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 11:51:52.898802 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:52.898699    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 11:51:53.887100 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:53.887072    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:53.887612 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:53.887111    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:53.887612 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:53.887215    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:53.887612 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:53.887306    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:54.092589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:54.092555    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92" event={"ID":"61c2ef99-6212-41d2-aac3-2c6ef1f04338","Type":"ContainerStarted","Data":"3107b0ab405ef5212cbf24df655101f887f4de7bc596a911512b45749ced8211"}
Sep 10 11:51:54.095717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:54.095692    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"dfca3e3b02f39cb95ead62863f8435195f198c49af4f99cca79d00e28c1fa27e"}
Sep 10 11:51:54.115082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:54.115046    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r2h92" podStartSLOduration=4.254314755 podStartE2EDuration="24.115032359s" totalImagesPullingTime="19.860717604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:54.114871581 +0000 UTC m=+24.718146493" watchObservedRunningTime="2026-09-10 11:51:54.115032359 +0000 UTC m=+24.718307319"
Sep 10 11:51:54.887168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:54.887127    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:54.887654 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:54.887284    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:55.887031 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:55.886996    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:55.887213 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:55.887008    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:55.887213 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:55.887129    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:55.887581 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:55.887211    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:56.886388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:56.886224    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:56.886531 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:56.886464    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:57.104030 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.103959    2571 generic.go:356] "Generic (PLEG): container finished" podID="c50f797b-4d46-458f-b5b0-f1d143a40e92" containerID="84c6a4bcff4d65f6d38a97af9ebcd653452befc0071458e61d2038e5685692ca" exitCode=0
Sep 10 11:51:57.104490 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.104033    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerDied","Data":"84c6a4bcff4d65f6d38a97af9ebcd653452befc0071458e61d2038e5685692ca"}
Sep 10 11:51:57.108105 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.108084    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" event={"ID":"0871f625-b780-4b5c-b20e-0e58dbc1c9e7","Type":"ContainerStarted","Data":"f3ec903ecbf50673d9fae5373032c0dbd3d84dd817ae241cf0ec3618ce9dfb06"}
Sep 10 11:51:57.108374 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.108353    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:57.108482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.108422    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:57.108482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.108468    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:57.122972 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.122945    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:57.123086 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.123074    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:51:57.157765 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.157726    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4" podStartSLOduration=9.58819245 podStartE2EDuration="27.157713591s" totalImagesPullingTime="17.569521141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:51:57.157534375 +0000 UTC m=+27.760809286" watchObservedRunningTime="2026-09-10 11:51:57.157713591 +0000 UTC m=+27.760988502"
Sep 10 11:51:57.886308 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.886278    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:57.886531 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:57.886279    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:57.886531 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:57.886382    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:57.886819 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:57.886790    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:58.374604 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:58.374575    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4j9sk"]
Sep 10 11:51:58.375136 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:58.374712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:58.375136 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:58.374821    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:58.380117 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:58.380076    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkxp2"]
Sep 10 11:51:58.380256 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:58.380208    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:58.380311 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:58.380293    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:51:58.380787 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:58.380768    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cfwkr"]
Sep 10 11:51:58.380874 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:58.380861    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:58.380987 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:58.380964    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:59.114843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:59.114812    2571 generic.go:356] "Generic (PLEG): container finished" podID="c50f797b-4d46-458f-b5b0-f1d143a40e92" containerID="97d2b262c468990e9ea04cd264296baab8d57f335f496d75661bb0044a5c70e5" exitCode=0
Sep 10 11:51:59.115028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:59.114891    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerDied","Data":"97d2b262c468990e9ea04cd264296baab8d57f335f496d75661bb0044a5c70e5"}
Sep 10 11:51:59.887590 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:59.887564    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:51:59.887989 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:59.887659    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:51:59.887989 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:59.887684    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:51:59.887989 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:51:59.887712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:51:59.887989 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:59.887812    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:51:59.887989 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:51:59.887927    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:52:01.121839 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:01.121752    2571 generic.go:356] "Generic (PLEG): container finished" podID="c50f797b-4d46-458f-b5b0-f1d143a40e92" containerID="72b72813accb3b882cc4f20d3e82c8535220f1b23f32169954a2b7fa86cea0a7" exitCode=0
Sep 10 11:52:01.121839 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:01.121817    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerDied","Data":"72b72813accb3b882cc4f20d3e82c8535220f1b23f32169954a2b7fa86cea0a7"}
Sep 10 11:52:01.886431 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:01.886399    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:52:01.886633 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:01.886399    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:01.886633 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:01.886535    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cfwkr" podUID="6eb31aad-f7b6-456d-a8dd-717ea539a6e1"
Sep 10 11:52:01.886633 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:01.886568    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:52:01.886788 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:01.886651    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4j9sk" podUID="ec47f958-f950-4af7-8a0e-138f8dce501f"
Sep 10 11:52:01.886788 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:01.886734    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkxp2" podUID="92c58a92-66b4-4673-887e-b84c82a92d80"
Sep 10 11:52:03.171323 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.171293    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-41.ec2.internal" event="NodeReady"
Sep 10 11:52:03.171693 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.171418    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 11:52:03.212280 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.212244    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-cd6cfd846-t2kx6"]
Sep 10 11:52:03.229013 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.228984    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4z47p"]
Sep 10 11:52:03.229192 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.229124    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.232793 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.232594    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 11:52:03.232793 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.232636    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 11:52:03.233890 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.233870    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sbl2s\""
Sep 10 11:52:03.234013 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.233901    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 11:52:03.239270 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.239241    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cd6cfd846-t2kx6"]
Sep 10 11:52:03.239374 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.239292    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f4dmp"]
Sep 10 11:52:03.239600 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.239518    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 11:52:03.239600 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.239538    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.249088 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.249063    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 11:52:03.249204 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.249063    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 11:52:03.249204 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.249126    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2c8qf\""
Sep 10 11:52:03.249204 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.249181    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 11:52:03.255521 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.255504    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4z47p"]
Sep 10 11:52:03.255596 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.255579    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.258719 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.258699    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 11:52:03.259011 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.258734    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q76h9\""
Sep 10 11:52:03.259011 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.258754    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 11:52:03.259664 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.259632    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f4dmp"]
Sep 10 11:52:03.356016 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.355983    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-config-volume\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.356016 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356016    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe9a7703-a40d-4b5d-91dd-5ff448d80136-cert\") pod \"ingress-canary-4z47p\" (UID: \"fe9a7703-a40d-4b5d-91dd-5ff448d80136\") " pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.356247 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356035    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-certificates\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356247 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356094    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-tmp-dir\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.356247 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356213    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9xq4\" (UniqueName: \"kubernetes.io/projected/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-kube-api-access-k9xq4\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.356247 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356239    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-image-registry-private-configuration\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356393 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-installation-pull-secrets\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356393 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356326    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzmzj\" (UniqueName: \"kubernetes.io/projected/fe9a7703-a40d-4b5d-91dd-5ff448d80136-kube-api-access-mzmzj\") pod \"ingress-canary-4z47p\" (UID: \"fe9a7703-a40d-4b5d-91dd-5ff448d80136\") " pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.356393 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356342    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fcf8bcfc-cb1d-4a39-960b-95070391d366-ca-trust-extracted\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356393 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356363    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx4s9\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-kube-api-access-lx4s9\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356393 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356388    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.356561 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356561 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356424    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-trusted-ca\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.356561 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.356441    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-bound-sa-token\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457355 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457324    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9xq4\" (UniqueName: \"kubernetes.io/projected/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-kube-api-access-k9xq4\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.457552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457363    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-image-registry-private-configuration\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457507    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-installation-pull-secrets\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzmzj\" (UniqueName: \"kubernetes.io/projected/fe9a7703-a40d-4b5d-91dd-5ff448d80136-kube-api-access-mzmzj\") pod \"ingress-canary-4z47p\" (UID: \"fe9a7703-a40d-4b5d-91dd-5ff448d80136\") " pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fcf8bcfc-cb1d-4a39-960b-95070391d366-ca-trust-extracted\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457596    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx4s9\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-kube-api-access-lx4s9\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457673    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-trusted-ca\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457703    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-bound-sa-token\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.457718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457733    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-config-volume\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.458093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457798    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe9a7703-a40d-4b5d-91dd-5ff448d80136-cert\") pod \"ingress-canary-4z47p\" (UID: \"fe9a7703-a40d-4b5d-91dd-5ff448d80136\") " pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.458093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457827    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-certificates\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.458093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.457852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-tmp-dir\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.458352 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.458190    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-tmp-dir\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.458352 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.458285    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:03.458352 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.458308    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fcf8bcfc-cb1d-4a39-960b-95070391d366-ca-trust-extracted\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.458502 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.458361    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:03.95833298 +0000 UTC m=+34.561607884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:03.459149 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.458734    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:03.459149 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.458756    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:03.459149 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.458817    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:03.958796189 +0000 UTC m=+34.562071091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:03.459149 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.458911    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-config-volume\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.459149 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.459108    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-certificates\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.459753 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.459719    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-trusted-ca\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.462409 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.462354    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-installation-pull-secrets\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.462409 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.462366    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-image-registry-private-configuration\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.462409 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.462379    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe9a7703-a40d-4b5d-91dd-5ff448d80136-cert\") pod \"ingress-canary-4z47p\" (UID: \"fe9a7703-a40d-4b5d-91dd-5ff448d80136\") " pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.470147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.470127    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9xq4\" (UniqueName: \"kubernetes.io/projected/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-kube-api-access-k9xq4\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.472294 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.472268    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzmzj\" (UniqueName: \"kubernetes.io/projected/fe9a7703-a40d-4b5d-91dd-5ff448d80136-kube-api-access-mzmzj\") pod \"ingress-canary-4z47p\" (UID: \"fe9a7703-a40d-4b5d-91dd-5ff448d80136\") " pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.472467 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.472451    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx4s9\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-kube-api-access-lx4s9\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.472723 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.472708    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-bound-sa-token\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.549520 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.549493    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4z47p"
Sep 10 11:52:03.558781 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.558749    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:52:03.558886 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.558829    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:52:03.558925 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.558892    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:52:03.558925 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.558920    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:52:03.558989 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.558952    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:35.558936736 +0000 UTC m=+66.162211625 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:52:03.558989 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.558966    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret podName:92c58a92-66b4-4673-887e-b84c82a92d80 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:35.558960009 +0000 UTC m=+66.162234898 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret") pod "global-pull-secret-syncer-nkxp2" (UID: "92c58a92-66b4-4673-887e-b84c82a92d80") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:52:03.756587 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.756360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:03.760048 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.760014    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:03.760341 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.760316    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:52:03.760429 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.760352    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:52:03.760429 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.760363    2571 projected.go:196] Error preparing data for projected volume kube-api-access-6vs94 for pod openshift-network-diagnostics/network-check-target-4j9sk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:52:03.760429 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.760422    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94 podName:ec47f958-f950-4af7-8a0e-138f8dce501f nodeName:}" failed. No retries permitted until 2026-09-10 11:52:35.760402892 +0000 UTC m=+66.363677806 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6vs94" (UniqueName: "kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94") pod "network-check-target-4j9sk" (UID: "ec47f958-f950-4af7-8a0e-138f8dce501f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:52:03.849819 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.849787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:03.855731 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.855698    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4z47p"]
Sep 10 11:52:03.859425 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:52:03.859397    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe9a7703_a40d_4b5d_91dd_5ff448d80136.slice/crio-d809ba22570561b0772331e378612907ede988b4c27ae31caeb16b7812546fdb WatchSource:0}: Error finding container d809ba22570561b0772331e378612907ede988b4c27ae31caeb16b7812546fdb: Status 404 returned error can't find the container with id d809ba22570561b0772331e378612907ede988b4c27ae31caeb16b7812546fdb
Sep 10 11:52:03.886537 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.886489    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:52:03.886537 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.886519    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:03.886705 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.886489    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:52:03.890126 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.889896    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 11:52:03.890126 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.889994    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 11:52:03.890126 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.890099    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-69fjj\""
Sep 10 11:52:03.891323 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.890729    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 11:52:03.891323 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.890887    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 11:52:03.892336 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.890831    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zrvwr\""
Sep 10 11:52:03.961236 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.961210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:03.961335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:03.961243    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:03.961384 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.961358    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:03.961426 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.961414    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:04.961396748 +0000 UTC m=+35.564671642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:03.961477 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.961362    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:03.961477 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.961447    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:03.961546 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:03.961491    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:04.961478833 +0000 UTC m=+35.564753750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:04.131373 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:04.131300    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4z47p" event={"ID":"fe9a7703-a40d-4b5d-91dd-5ff448d80136","Type":"ContainerStarted","Data":"d809ba22570561b0772331e378612907ede988b4c27ae31caeb16b7812546fdb"}
Sep 10 11:52:04.970512 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:04.970472    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:04.970990 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:04.970521    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:04.970990 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:04.970635    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:04.970990 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:04.970648    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:04.970990 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:04.970703    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:06.970688725 +0000 UTC m=+37.573963614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:04.970990 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:04.970634    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:04.970990 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:04.970776    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:06.970757591 +0000 UTC m=+37.574032498 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:06.986416 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:06.986377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:06.986982 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:06.986435    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:06.986982 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:06.986548    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:06.986982 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:06.986566    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:06.986982 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:06.986583    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:06.986982 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:06.986626    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:10.986605149 +0000 UTC m=+41.589880042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:06.986982 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:06.986645    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:10.986635571 +0000 UTC m=+41.589910462 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:08.142189 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:08.142093    2571 generic.go:356] "Generic (PLEG): container finished" podID="c50f797b-4d46-458f-b5b0-f1d143a40e92" containerID="814470b97383cf7ad4ebe81d8e5fd94600888f19456dd0eccf475412abac5022" exitCode=0
Sep 10 11:52:08.142189 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:08.142174    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerDied","Data":"814470b97383cf7ad4ebe81d8e5fd94600888f19456dd0eccf475412abac5022"}
Sep 10 11:52:08.143651 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:08.143629    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4z47p" event={"ID":"fe9a7703-a40d-4b5d-91dd-5ff448d80136","Type":"ContainerStarted","Data":"4c189e5113f95155d21f114bdffa23e433882930e278b1a6b854bc534f2f037f"}
Sep 10 11:52:08.187617 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:08.187572    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4z47p" podStartSLOduration=1.319165254 podStartE2EDuration="5.187557692s" totalImagesPullingTime="3.868392438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:52:08.186605288 +0000 UTC m=+38.789880191" watchObservedRunningTime="2026-09-10 11:52:08.187557692 +0000 UTC m=+38.790832603"
Sep 10 11:52:08.941540 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:08.941348    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:09.039191 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:09.039140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:09.139223 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:09.139147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:09.148836 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:09.148809    2571 generic.go:356] "Generic (PLEG): container finished" podID="c50f797b-4d46-458f-b5b0-f1d143a40e92" containerID="a432cc2a53a17abedf0bbec780c68c9f277385fa5f279fa8e585d67b103c4ffc" exitCode=0
Sep 10 11:52:09.149197 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:09.148884    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerDied","Data":"a432cc2a53a17abedf0bbec780c68c9f277385fa5f279fa8e585d67b103c4ffc"}
Sep 10 11:52:10.154462 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:10.154425    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" event={"ID":"c50f797b-4d46-458f-b5b0-f1d143a40e92","Type":"ContainerStarted","Data":"c5c582892c452a59caab269ca65d6f2d1815a990eec3810b50d4de82247796fe"}
Sep 10 11:52:10.178786 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:10.178742    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lt6jn" podStartSLOduration=8.196867729 podStartE2EDuration="40.178726579s" totalImagesPullingTime="31.98185885s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:52:10.177374091 +0000 UTC m=+40.780649002" watchObservedRunningTime="2026-09-10 11:52:10.178726579 +0000 UTC m=+40.782001490"
Sep 10 11:52:10.308135 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:10.308110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:10.408773 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:10.408703    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:10.816887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:10.816857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:10.878653 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:10.878615    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:11.018868 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:11.018835    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:11.019007 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:11.018875    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:11.019007 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:11.018969    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:11.019111 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:11.019032    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:19.019017223 +0000 UTC m=+49.622292112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:11.019111 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:11.019031    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:11.019111 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:11.019050    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:11.019111 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:11.019099    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:19.019080944 +0000 UTC m=+49.622355837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:11.969375 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:11.969339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:12.044820 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:12.044795    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:17.355128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:17.355080    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:17.434635 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:17.434606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:19.075862 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:19.075823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:19.075862 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:19.075863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:19.076299 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:19.075978    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:19.076299 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:19.075996    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:19.076299 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:19.076008    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:19.076299 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:19.076046    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:35.076030238 +0000 UTC m=+65.679305132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:19.076299 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:19.076061    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:35.076055178 +0000 UTC m=+65.679330067 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:19.093090 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:19.093065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:19.166717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:19.166694    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:29.132574 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:29.132543    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9p6c4"
Sep 10 11:52:29.298474 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:29.298440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:29.462044 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:29.462011    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:32.949170 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:32.949127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:33.049076 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:33.049048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:35.081643 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.081599    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:52:35.081643 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.081644    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:52:35.082222 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.081763    2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:52:35.082222 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.081775    2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-cd6cfd846-t2kx6: secret "image-registry-tls" not found
Sep 10 11:52:35.082222 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.081817    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls podName:fcf8bcfc-cb1d-4a39-960b-95070391d366 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:07.08180294 +0000 UTC m=+97.685077830 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls") pod "image-registry-cd6cfd846-t2kx6" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366") : secret "image-registry-tls" not found
Sep 10 11:52:35.082222 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.081762    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:52:35.082222 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.081915    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls podName:738ddeb2-5f6a-4545-bc93-b7bcca8e3836 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:07.081895824 +0000 UTC m=+97.685170736 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls") pod "dns-default-f4dmp" (UID: "738ddeb2-5f6a-4545-bc93-b7bcca8e3836") : secret "dns-default-metrics-tls" not found
Sep 10 11:52:35.585876 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.585839    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:52:35.586062 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.585887    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:52:35.588915 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.588893    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 11:52:35.589030 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.588920    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 11:52:35.596993 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.596973    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 11:52:35.597044 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:35.597030    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs podName:6eb31aad-f7b6-456d-a8dd-717ea539a6e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:39.597014799 +0000 UTC m=+130.200289688 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs") pod "network-metrics-daemon-cfwkr" (UID: "6eb31aad-f7b6-456d-a8dd-717ea539a6e1") : secret "metrics-daemon-secret" not found
Sep 10 11:52:35.599954 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.599928    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/92c58a92-66b4-4673-887e-b84c82a92d80-original-pull-secret\") pod \"global-pull-secret-syncer-nkxp2\" (UID: \"92c58a92-66b4-4673-887e-b84c82a92d80\") " pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:52:35.699756 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.699726    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkxp2"
Sep 10 11:52:35.787928 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.787882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:35.790730 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.790713    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 11:52:35.801437 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.801417    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 11:52:35.812120 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.812097    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vs94\" (UniqueName: \"kubernetes.io/projected/ec47f958-f950-4af7-8a0e-138f8dce501f-kube-api-access-6vs94\") pod \"network-check-target-4j9sk\" (UID: \"ec47f958-f950-4af7-8a0e-138f8dce501f\") " pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:35.964780 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:35.964744    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkxp2"]
Sep 10 11:52:35.968293 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:52:35.968269    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c58a92_66b4_4673_887e_b84c82a92d80.slice/crio-6b5a3f8e8118eaba1fd9239629fbbf288967b6bf73ec85686cff25a013594287 WatchSource:0}: Error finding container 6b5a3f8e8118eaba1fd9239629fbbf288967b6bf73ec85686cff25a013594287: Status 404 returned error can't find the container with id 6b5a3f8e8118eaba1fd9239629fbbf288967b6bf73ec85686cff25a013594287
Sep 10 11:52:36.016474 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:36.016456    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-69fjj\""
Sep 10 11:52:36.024066 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:36.024051    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:36.218161 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:36.218076    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkxp2" event={"ID":"92c58a92-66b4-4673-887e-b84c82a92d80","Type":"ContainerStarted","Data":"6b5a3f8e8118eaba1fd9239629fbbf288967b6bf73ec85686cff25a013594287"}
Sep 10 11:52:36.293462 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:36.293434    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4j9sk"]
Sep 10 11:52:36.296928 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:52:36.296905    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec47f958_f950_4af7_8a0e_138f8dce501f.slice/crio-f9b3b70a40daf27b6f8d9f643ff1c3e2d17214f56e56c57a10a313d2143d5115 WatchSource:0}: Error finding container f9b3b70a40daf27b6f8d9f643ff1c3e2d17214f56e56c57a10a313d2143d5115: Status 404 returned error can't find the container with id f9b3b70a40daf27b6f8d9f643ff1c3e2d17214f56e56c57a10a313d2143d5115
Sep 10 11:52:37.223443 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:37.223403    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4j9sk" event={"ID":"ec47f958-f950-4af7-8a0e-138f8dce501f","Type":"ContainerStarted","Data":"f9b3b70a40daf27b6f8d9f643ff1c3e2d17214f56e56c57a10a313d2143d5115"}
Sep 10 11:52:39.320192 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:39.320147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:39.418429 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:39.418394    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:39.483553 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:39.483520    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:41.235362 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:41.235272    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4j9sk" event={"ID":"ec47f958-f950-4af7-8a0e-138f8dce501f","Type":"ContainerStarted","Data":"388b35a46ca226758340bdd82711f8679a8626e1bd960426b8ce80790a09eda3"}
Sep 10 11:52:41.235778 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:41.235421    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:52:41.236832 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:41.236810    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkxp2" event={"ID":"92c58a92-66b4-4673-887e-b84c82a92d80","Type":"ContainerStarted","Data":"c3bfd33077f04ce4bd733470c44e4a51eccc6a0ba3f8a5d6af5764f88bd29ebe"}
Sep 10 11:52:41.253835 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:41.253796    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4j9sk" podStartSLOduration=66.612476895 podStartE2EDuration="1m11.253784781s" totalImagesPullingTime="4.641307886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:52:41.252710492 +0000 UTC m=+71.855985402" watchObservedRunningTime="2026-09-10 11:52:41.253784781 +0000 UTC m=+71.857059692"
Sep 10 11:52:43.453451 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:43.453419    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:43.522017 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:43.521991    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:45.747823 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.747743    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nkxp2" podStartSLOduration=70.774647609 podStartE2EDuration="1m15.747719394s" totalImagesPullingTime="4.973071785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:52:41.26756229 +0000 UTC m=+71.870837203" watchObservedRunningTime="2026-09-10 11:52:45.747719394 +0000 UTC m=+76.350994307"
Sep 10 11:52:45.749339 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.749312    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"]
Sep 10 11:52:45.751511 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.751490    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"]
Sep 10 11:52:45.751623 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.751604    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.753167 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.753132    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:45.753636 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.753617    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 11:52:45.753865 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.753843    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 11:52:45.754008 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.753994    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 11:52:45.754713 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.754697    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cztj8\""
Sep 10 11:52:45.755474 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.755448    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px4sj\" (UniqueName: \"kubernetes.io/projected/1ff61920-5eac-4ed5-9966-f532faabe202-kube-api-access-px4sj\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:45.755552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.755522    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/04d8a797-3778-4a8c-becf-401dd09db8d3-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.755612 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.755546    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.755665 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.755606    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 11:52:45.755665 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.755614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvtnk\" (UniqueName: \"kubernetes.io/projected/04d8a797-3778-4a8c-becf-401dd09db8d3-kube-api-access-fvtnk\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.755762 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.755680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:45.756447 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.756433    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 11:52:45.757267 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.757249    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:52:45.757774 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.757758    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-djq8f\""
Sep 10 11:52:45.758314 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.758268    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 11:52:45.772341 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.772314    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"]
Sep 10 11:52:45.773107 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.773090    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"]
Sep 10 11:52:45.832143 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.832114    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"]
Sep 10 11:52:45.833810 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.833792    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.837275 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.837257    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 11:52:45.837637 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.837619    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:52:45.837687 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.837664    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-c4lbj\""
Sep 10 11:52:45.837895 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.837882    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 11:52:45.837967 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.837883    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 11:52:45.856331 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvtnk\" (UniqueName: \"kubernetes.io/projected/04d8a797-3778-4a8c-becf-401dd09db8d3-kube-api-access-fvtnk\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.856427 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856339    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-config\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.856427 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856371    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-serving-cert\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.856427 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:45.856542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856427    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrnjk\" (UniqueName: \"kubernetes.io/projected/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-kube-api-access-jrnjk\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.856542 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:45.856455    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 11:52:45.856542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856473    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px4sj\" (UniqueName: \"kubernetes.io/projected/1ff61920-5eac-4ed5-9966-f532faabe202-kube-api-access-px4sj\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:45.856542 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:45.856509    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls podName:1ff61920-5eac-4ed5-9966-f532faabe202 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:46.356493467 +0000 UTC m=+76.959768357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jssvm" (UID: "1ff61920-5eac-4ed5-9966-f532faabe202") : secret "samples-operator-tls" not found
Sep 10 11:52:45.856729 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/04d8a797-3778-4a8c-becf-401dd09db8d3-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.856729 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.856568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.856729 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:45.856657    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:45.856729 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:45.856693    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls podName:04d8a797-3778-4a8c-becf-401dd09db8d3 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:46.356681681 +0000 UTC m=+76.959956569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nkzjm" (UID: "04d8a797-3778-4a8c-becf-401dd09db8d3") : secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:45.857342 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.857314    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/04d8a797-3778-4a8c-becf-401dd09db8d3-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.858059 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.858038    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"]
Sep 10 11:52:45.868579 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.868554    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px4sj\" (UniqueName: \"kubernetes.io/projected/1ff61920-5eac-4ed5-9966-f532faabe202-kube-api-access-px4sj\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:45.872567 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.872550    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvtnk\" (UniqueName: \"kubernetes.io/projected/04d8a797-3778-4a8c-becf-401dd09db8d3-kube-api-access-fvtnk\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:45.957671 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.957638    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-serving-cert\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.957840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.957678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrnjk\" (UniqueName: \"kubernetes.io/projected/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-kube-api-access-jrnjk\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.957905 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.957883    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-config\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.958408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.958386    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-config\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.959704 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.959678    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-serving-cert\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:45.965751 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:45.965728    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrnjk\" (UniqueName: \"kubernetes.io/projected/ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab-kube-api-access-jrnjk\") pod \"service-ca-operator-859f94c4b-4mt7m\" (UID: \"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:46.035705 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:46.035588    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:46.142176 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:46.142131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"
Sep 10 11:52:46.150554 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:46.150495    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:46.360825 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:46.360739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:46.360825 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:46.360809    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:46.361035 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:46.360895    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:46.361035 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:46.360924    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 11:52:46.361035 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:46.360965    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls podName:04d8a797-3778-4a8c-becf-401dd09db8d3 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:47.36094763 +0000 UTC m=+77.964222534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nkzjm" (UID: "04d8a797-3778-4a8c-becf-401dd09db8d3") : secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:46.361035 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:46.360980    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls podName:1ff61920-5eac-4ed5-9966-f532faabe202 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:47.360973077 +0000 UTC m=+77.964247966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jssvm" (UID: "1ff61920-5eac-4ed5-9966-f532faabe202") : secret "samples-operator-tls" not found
Sep 10 11:52:46.410254 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:46.410227    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m"]
Sep 10 11:52:46.414500 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:52:46.414473    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf4b6cd_c1fb_40b4_a44a_af1ab7ed12ab.slice/crio-fe4cebeb3fba1355739061e05b0552bd2aadc0f2c6beed7b23904d26c29b9307 WatchSource:0}: Error finding container fe4cebeb3fba1355739061e05b0552bd2aadc0f2c6beed7b23904d26c29b9307: Status 404 returned error can't find the container with id fe4cebeb3fba1355739061e05b0552bd2aadc0f2c6beed7b23904d26c29b9307
Sep 10 11:52:47.253035 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:47.252993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m" event={"ID":"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab","Type":"ContainerStarted","Data":"fe4cebeb3fba1355739061e05b0552bd2aadc0f2c6beed7b23904d26c29b9307"}
Sep 10 11:52:47.367581 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:47.367539    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:47.367747 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:47.367639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:47.367747 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:47.367717    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:47.367874 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:47.367747    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 11:52:47.367874 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:47.367798    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls podName:04d8a797-3778-4a8c-becf-401dd09db8d3 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:49.367777355 +0000 UTC m=+79.971052267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nkzjm" (UID: "04d8a797-3778-4a8c-becf-401dd09db8d3") : secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:47.367874 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:47.367816    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls podName:1ff61920-5eac-4ed5-9966-f532faabe202 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:49.367808669 +0000 UTC m=+79.971083557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jssvm" (UID: "1ff61920-5eac-4ed5-9966-f532faabe202") : secret "samples-operator-tls" not found
Sep 10 11:52:48.051727 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:48.051624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:49.259553 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:49.259516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m" event={"ID":"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab","Type":"ContainerStarted","Data":"677beca8803b35fb1145ded4145a274c7623ab4d592ab4d49b254f4416fce18a"}
Sep 10 11:52:49.282063 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:49.282015    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m" podStartSLOduration=2.156873306 podStartE2EDuration="4.28200188s" totalImagesPullingTime="2.125128574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:52:49.28095011 +0000 UTC m=+79.884225023" watchObservedRunningTime="2026-09-10 11:52:49.28200188 +0000 UTC m=+79.885276829"
Sep 10 11:52:49.381605 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:49.381553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:49.381774 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:49.381657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:49.381774 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:49.381715    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:49.381774 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:49.381764    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 11:52:49.381888 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:49.381785    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls podName:04d8a797-3778-4a8c-becf-401dd09db8d3 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:53.381769285 +0000 UTC m=+83.985044178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nkzjm" (UID: "04d8a797-3778-4a8c-becf-401dd09db8d3") : secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:49.381888 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:49.381828    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls podName:1ff61920-5eac-4ed5-9966-f532faabe202 nodeName:}" failed. No retries permitted until 2026-09-10 11:52:53.381814601 +0000 UTC m=+83.985089491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jssvm" (UID: "1ff61920-5eac-4ed5-9966-f532faabe202") : secret "samples-operator-tls" not found
Sep 10 11:52:50.395361 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.395326    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pcjzx"]
Sep 10 11:52:50.398502 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.398478    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.401784 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.401761    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 11:52:50.401869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.401764    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 11:52:50.401869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.401806    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 11:52:50.402477 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.402458    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 11:52:50.402564 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.402477    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s4qzc\""
Sep 10 11:52:50.405967 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.405948    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:50.418167 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.418129    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pcjzx"]
Sep 10 11:52:50.490628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.490576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4j8\" (UniqueName: \"kubernetes.io/projected/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-api-access-tf4j8\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.490628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.490640    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/671603a8-2ec0-4abd-ac85-cd068f477e7e-data-volume\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.490858 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.490718    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.490858 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.490751    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.490858 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.490785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/671603a8-2ec0-4abd-ac85-cd068f477e7e-crio-socket\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.497451 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.497428    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:50.591107 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591081    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.591107 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591110    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.591349 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591138    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/671603a8-2ec0-4abd-ac85-cd068f477e7e-crio-socket\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.591349 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591233    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/671603a8-2ec0-4abd-ac85-cd068f477e7e-crio-socket\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.591349 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:50.591276    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:50.591349 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591275    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4j8\" (UniqueName: \"kubernetes.io/projected/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-api-access-tf4j8\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.591349 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:50.591332    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:52:50.591561 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:50.591342    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:51.091322993 +0000 UTC m=+81.694597887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:50.591561 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:50.591402    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:51.091389311 +0000 UTC m=+81.694664213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : configmap "kube-rbac-proxy" not found
Sep 10 11:52:50.591561 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591448    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/671603a8-2ec0-4abd-ac85-cd068f477e7e-data-volume\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.591720 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.591704    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/671603a8-2ec0-4abd-ac85-cd068f477e7e-data-volume\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.604865 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.604838    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4j8\" (UniqueName: \"kubernetes.io/projected/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-api-access-tf4j8\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:50.897921 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:50.897887    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:51.095388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:51.095351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:51.095388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:51.095386    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:51.095614 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:51.095469    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:52:51.095614 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:51.095482    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:51.095614 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:51.095540    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:52.095523697 +0000 UTC m=+82.698798586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : configmap "kube-rbac-proxy" not found
Sep 10 11:52:51.095614 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:51.095561    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:52.095555059 +0000 UTC m=+82.698829948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:51.166549 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:51.166477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:51.266925 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:51.266893    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:52.103371 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:52.103244    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:52.103371 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:52.103289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:52.103937 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:52.103377    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:52:52.103937 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:52.103443    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:54.103427183 +0000 UTC m=+84.706702072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : configmap "kube-rbac-proxy" not found
Sep 10 11:52:52.103937 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:52.103510    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:52.103937 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:52.103580    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:54.103561619 +0000 UTC m=+84.706836533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:52.344003 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:52.343973    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v4ghr_89662635-b5c4-47b7-8e37-b30f4b58fda4/dns-node-resolver/0.log"
Sep 10 11:52:52.835522 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:52.835491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:52.910861 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:52.910831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:53.150552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.150478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:53.251015 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.250979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:53.342063 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.342037    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dxl2z_0b8668d6-db4b-43c5-8567-51fcb8429334/node-ca/0.log"
Sep 10 11:52:53.414751 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.414682    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:52:53.414880 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.414751    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:52:53.414880 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:53.414831    2571 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 11:52:53.414880 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:53.414844    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:53.414971 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:53.414896    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls podName:1ff61920-5eac-4ed5-9966-f532faabe202 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:01.414878196 +0000 UTC m=+92.018153084 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-jssvm" (UID: "1ff61920-5eac-4ed5-9966-f532faabe202") : secret "samples-operator-tls" not found
Sep 10 11:52:53.414971 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:53.414913    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls podName:04d8a797-3778-4a8c-becf-401dd09db8d3 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:01.414905954 +0000 UTC m=+92.018180846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nkzjm" (UID: "04d8a797-3778-4a8c-becf-401dd09db8d3") : secret "cluster-monitoring-operator-tls" not found
Sep 10 11:52:53.804741 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.804710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:53.944843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:53.944818    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4z47p_fe9a7703-a40d-4b5d-91dd-5ff448d80136/serve-healthcheck-canary/0.log"
Sep 10 11:52:54.121173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:54.121074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:54.121173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:54.121110    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:54.121357 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:54.121216    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:52:54.121357 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:54.121233    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:54.121357 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:54.121284    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:58.121266363 +0000 UTC m=+88.724541253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:54.121357 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:54.121301    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:52:58.121292301 +0000 UTC m=+88.724567194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : configmap "kube-rbac-proxy" not found
Sep 10 11:52:54.210506 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:54.210465    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:54.285940 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:54.285906    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:56.304575 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:56.304545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:56.534709 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:56.534677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:56.623560 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:56.623452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:56.874418 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:56.874341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:57.005825 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:57.005795    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:57.713270 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:57.713237    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:58.002060 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:58.001982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:58.154147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:58.154112    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:58.154147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:58.154146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:52:58.154366 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:58.154233    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:52:58.154366 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:58.154299    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:53:06.154283856 +0000 UTC m=+96.757558745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : configmap "kube-rbac-proxy" not found
Sep 10 11:52:58.154366 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:58.154326    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:58.154475 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:52:58.154388    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls podName:671603a8-2ec0-4abd-ac85-cd068f477e7e nodeName:}" failed. No retries permitted until 2026-09-10 11:53:06.154376298 +0000 UTC m=+96.757651187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pcjzx" (UID: "671603a8-2ec0-4abd-ac85-cd068f477e7e") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:52:58.219876 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:58.219843    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:58.611351 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:58.611319    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:58.903134 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:58.903038    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:59.213032 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:59.213001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:52:59.803386 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:59.803353    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:00.000002 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:52:59.999963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:00.107890 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:00.107811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:00.209143 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:00.209110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:00.403330 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:00.403254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:00.837614 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:00.837575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:00.931371 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:00.931342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:01.403772 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:01.403737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:01.478330 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:01.478276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:53:01.478492 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:01.478373    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:53:01.478492 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:53:01.478422    2571 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 11:53:01.478492 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:53:01.478486    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls podName:04d8a797-3778-4a8c-becf-401dd09db8d3 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:17.478471153 +0000 UTC m=+108.081746043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-nkzjm" (UID: "04d8a797-3778-4a8c-becf-401dd09db8d3") : secret "cluster-monitoring-operator-tls" not found
Sep 10 11:53:01.480799 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:01.480775    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ff61920-5eac-4ed5-9966-f532faabe202-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-jssvm\" (UID: \"1ff61920-5eac-4ed5-9966-f532faabe202\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:53:01.665935 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:01.665844    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"
Sep 10 11:53:01.938716 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:01.938690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm"]
Sep 10 11:53:02.293100 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:02.293015    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm" event={"ID":"1ff61920-5eac-4ed5-9966-f532faabe202","Type":"ContainerStarted","Data":"1768ad7536b3b64a269ae971fb750e32abd045c3f91d1b5ce1efb86f231b17e4"}
Sep 10 11:53:02.809496 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:02.809463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:02.878505 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:02.878471    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:03.004441 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:03.004407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:03.587605 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:03.587564    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:04.212632 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.212594    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:04.300650 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.300621    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm" event={"ID":"1ff61920-5eac-4ed5-9966-f532faabe202","Type":"ContainerStarted","Data":"77d5766a3e832e1f6e75c2fedbfd9574e59491076a7bcb0783129a82c910ab4a"}
Sep 10 11:53:04.300796 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.300666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm" event={"ID":"1ff61920-5eac-4ed5-9966-f532faabe202","Type":"ContainerStarted","Data":"fd6fc852798bd1f1f36dbf9dbf405f3adb082b67025f6f3fdf942369c0b0e372"}
Sep 10 11:53:04.320094 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.320051    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-jssvm" podStartSLOduration=17.539457933 podStartE2EDuration="19.320037079s" totalImagesPullingTime="1.780579146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:04.319476475 +0000 UTC m=+94.922751385" watchObservedRunningTime="2026-09-10 11:53:04.320037079 +0000 UTC m=+94.923312001"
Sep 10 11:53:04.406026 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.406000    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:04.605843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.605734    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:04.796789 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.796754    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:04.996573 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:04.996521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:06.210569 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.210533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:06.217921 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.217895    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:53:06.217988 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.217928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:53:06.218539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.218521    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/671603a8-2ec0-4abd-ac85-cd068f477e7e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:53:06.220293 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.220277    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/671603a8-2ec0-4abd-ac85-cd068f477e7e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pcjzx\" (UID: \"671603a8-2ec0-4abd-ac85-cd068f477e7e\") " pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:53:06.306990 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.306958    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pcjzx"
Sep 10 11:53:06.580336 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:06.580258    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pcjzx"]
Sep 10 11:53:06.583163 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:06.583125    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671603a8_2ec0_4abd_ac85_cd068f477e7e.slice/crio-af182976ffa07fb1fa0510bff7ef04c4f7c0a8da38a549f62040e1f799d53c1c WatchSource:0}: Error finding container af182976ffa07fb1fa0510bff7ef04c4f7c0a8da38a549f62040e1f799d53c1c: Status 404 returned error can't find the container with id af182976ffa07fb1fa0510bff7ef04c4f7c0a8da38a549f62040e1f799d53c1c
Sep 10 11:53:07.126499 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.126471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:53:07.126682 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.126510    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:53:07.130512 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.129539    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/738ddeb2-5f6a-4545-bc93-b7bcca8e3836-metrics-tls\") pod \"dns-default-f4dmp\" (UID: \"738ddeb2-5f6a-4545-bc93-b7bcca8e3836\") " pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:53:07.130512 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.129726    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"image-registry-cd6cfd846-t2kx6\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") " pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:53:07.142759 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.142742    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sbl2s\""
Sep 10 11:53:07.150658 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.150635    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:53:07.167144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.167124    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-q76h9\""
Sep 10 11:53:07.175296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.175275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:53:07.314469 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.314434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcjzx" event={"ID":"671603a8-2ec0-4abd-ac85-cd068f477e7e","Type":"ContainerStarted","Data":"143a2b998a237a3bfc0196a1b6ba876a2276748e78e30b72c110acb0f944764a"}
Sep 10 11:53:07.314818 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.314520    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcjzx" event={"ID":"671603a8-2ec0-4abd-ac85-cd068f477e7e","Type":"ContainerStarted","Data":"af182976ffa07fb1fa0510bff7ef04c4f7c0a8da38a549f62040e1f799d53c1c"}
Sep 10 11:53:07.430589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.430537    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-cd6cfd846-t2kx6"]
Sep 10 11:53:07.435548 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:07.435514    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf8bcfc_cb1d_4a39_960b_95070391d366.slice/crio-185b36b55e8bdf7c425e8a5c1c256deb9e8f701450ac43d0ae791ca3ed6e1b2a WatchSource:0}: Error finding container 185b36b55e8bdf7c425e8a5c1c256deb9e8f701450ac43d0ae791ca3ed6e1b2a: Status 404 returned error can't find the container with id 185b36b55e8bdf7c425e8a5c1c256deb9e8f701450ac43d0ae791ca3ed6e1b2a
Sep 10 11:53:07.458769 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.458742    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f4dmp"]
Sep 10 11:53:07.462142 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:07.462111    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738ddeb2_5f6a_4545_bc93_b7bcca8e3836.slice/crio-1191c37b17259d6c81c4ac1303de1ba6ebd0d99e2ea67fb8a08806edc959664c WatchSource:0}: Error finding container 1191c37b17259d6c81c4ac1303de1ba6ebd0d99e2ea67fb8a08806edc959664c: Status 404 returned error can't find the container with id 1191c37b17259d6c81c4ac1303de1ba6ebd0d99e2ea67fb8a08806edc959664c
Sep 10 11:53:07.478952 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.478921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:07.580236 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:07.580200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:08.320117 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:08.320024    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" event={"ID":"fcf8bcfc-cb1d-4a39-960b-95070391d366","Type":"ContainerStarted","Data":"0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc"}
Sep 10 11:53:08.320117 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:08.320083    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" event={"ID":"fcf8bcfc-cb1d-4a39-960b-95070391d366","Type":"ContainerStarted","Data":"185b36b55e8bdf7c425e8a5c1c256deb9e8f701450ac43d0ae791ca3ed6e1b2a"}
Sep 10 11:53:08.320117 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:08.320111    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:53:08.321350 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:08.321323    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f4dmp" event={"ID":"738ddeb2-5f6a-4545-bc93-b7bcca8e3836","Type":"ContainerStarted","Data":"1191c37b17259d6c81c4ac1303de1ba6ebd0d99e2ea67fb8a08806edc959664c"}
Sep 10 11:53:08.323238 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:08.323210    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcjzx" event={"ID":"671603a8-2ec0-4abd-ac85-cd068f477e7e","Type":"ContainerStarted","Data":"f47d1c4e9e197603a0ed585e3b62cd59bb9857a5ff261d26084eb97dc938ee88"}
Sep 10 11:53:08.344839 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:08.344792    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" podStartSLOduration=98.34477157 podStartE2EDuration="1m38.34477157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:08.344587511 +0000 UTC m=+98.947862422" watchObservedRunningTime="2026-09-10 11:53:08.34477157 +0000 UTC m=+98.948046483"
Sep 10 11:53:09.049967 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:09.049941    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:09.118784 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:09.118756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:09.328128 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:09.328093    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pcjzx" event={"ID":"671603a8-2ec0-4abd-ac85-cd068f477e7e","Type":"ContainerStarted","Data":"40300fb37d2d152b10fc664186d7a611eac5c158d42abcb62fee05ad1fa015be"}
Sep 10 11:53:09.349399 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:09.349350    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pcjzx" podStartSLOduration=17.140067617 podStartE2EDuration="19.349332144s" totalImagesPullingTime="2.209264527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:52:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:09.347920262 +0000 UTC m=+99.951195172" watchObservedRunningTime="2026-09-10 11:53:09.349332144 +0000 UTC m=+99.952607054"
Sep 10 11:53:10.754579 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:10.754549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:10.855239 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:10.855204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:10.925069 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:10.925038    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:11.051172 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.051131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:11.338116 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.338046    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f4dmp" event={"ID":"738ddeb2-5f6a-4545-bc93-b7bcca8e3836","Type":"ContainerStarted","Data":"8a2fd6ab028421331f3bf60a46853c7665caf43ad5751e3f0ae842415dfb5045"}
Sep 10 11:53:11.338256 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.338119    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f4dmp" event={"ID":"738ddeb2-5f6a-4545-bc93-b7bcca8e3836","Type":"ContainerStarted","Data":"b895842ddf3180cd86587ac678446d91c856ddc1156ebe26427cea88e157f689"}
Sep 10 11:53:11.338256 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.338180    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:53:11.359769 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.359667    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f4dmp" podStartSLOduration=64.835937884 podStartE2EDuration="1m8.359652041s" totalImagesPullingTime="3.523714157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:11.358767621 +0000 UTC m=+101.962042534" watchObservedRunningTime="2026-09-10 11:53:11.359652041 +0000 UTC m=+101.962926954"
Sep 10 11:53:11.436049 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.436021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:11.510050 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:11.510021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:12.236514 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:12.236477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:12.241286 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:12.241264    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4j9sk"
Sep 10 11:53:12.307821 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:12.307796    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:12.446972 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:12.446937    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:12.648053 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:12.647967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:12.895210 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:12.895180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:17.501821 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.501776    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:53:17.504141 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.504119    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/04d8a797-3778-4a8c-becf-401dd09db8d3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-nkzjm\" (UID: \"04d8a797-3778-4a8c-becf-401dd09db8d3\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:53:17.561167 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.561112    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"
Sep 10 11:53:17.831169 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.831082    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm"]
Sep 10 11:53:17.833911 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:17.833884    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d8a797_3778_4a8c_becf_401dd09db8d3.slice/crio-1118ec57c0cefde118b954bf1ee830acbe513febbb0b62e48a4e2569378922a4 WatchSource:0}: Error finding container 1118ec57c0cefde118b954bf1ee830acbe513febbb0b62e48a4e2569378922a4: Status 404 returned error can't find the container with id 1118ec57c0cefde118b954bf1ee830acbe513febbb0b62e48a4e2569378922a4
Sep 10 11:53:17.975240 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.975207    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"]
Sep 10 11:53:17.977589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.977568    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:17.981738 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.981718    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 11:53:17.981995 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.981960    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 11:53:17.982064 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.982003    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 11:53:17.982519 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.982493    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 11:53:17.982632 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.982520    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 11:53:17.982742 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.982728    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 11:53:17.988074 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.988056    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 11:53:17.997167 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:17.997135    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"]
Sep 10 11:53:18.107099 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.107011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f20e59-9d14-4441-b3ef-83a908e64e3b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.107099 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.107061    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-hub\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.107292 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.107116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.107292 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.107138    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4v6h\" (UniqueName: \"kubernetes.io/projected/37f20e59-9d14-4441-b3ef-83a908e64e3b-kube-api-access-j4v6h\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.107292 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.107192    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-ca\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.107292 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.107228    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.207936 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.207901    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.207936 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.207938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4v6h\" (UniqueName: \"kubernetes.io/projected/37f20e59-9d14-4441-b3ef-83a908e64e3b-kube-api-access-j4v6h\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.208140 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.207967    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-ca\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.208140 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.207989    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.208140 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.208020    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f20e59-9d14-4441-b3ef-83a908e64e3b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.208140 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.208057    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-hub\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.208813 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.208789    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/37f20e59-9d14-4441-b3ef-83a908e64e3b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.210475 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.210450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-ca\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.210604 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.210581    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.210861 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.210843    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-hub\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.210933 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.210917    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/37f20e59-9d14-4441-b3ef-83a908e64e3b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.227748 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.227723    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4v6h\" (UniqueName: \"kubernetes.io/projected/37f20e59-9d14-4441-b3ef-83a908e64e3b-kube-api-access-j4v6h\") pod \"cluster-proxy-proxy-agent-5895744555-cnznw\" (UID: \"37f20e59-9d14-4441-b3ef-83a908e64e3b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.296082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.296044    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"
Sep 10 11:53:18.361075 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.360996    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm" event={"ID":"04d8a797-3778-4a8c-becf-401dd09db8d3","Type":"ContainerStarted","Data":"1118ec57c0cefde118b954bf1ee830acbe513febbb0b62e48a4e2569378922a4"}
Sep 10 11:53:18.569882 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:18.569852    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw"]
Sep 10 11:53:18.573186 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:18.573139    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37f20e59_9d14_4441_b3ef_83a908e64e3b.slice/crio-756a8f5f1694c79a35c384378bf934b1ed069b81ac185d2fab4df683f7731a6d WatchSource:0}: Error finding container 756a8f5f1694c79a35c384378bf934b1ed069b81ac185d2fab4df683f7731a6d: Status 404 returned error can't find the container with id 756a8f5f1694c79a35c384378bf934b1ed069b81ac185d2fab4df683f7731a6d
Sep 10 11:53:19.366869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:19.366829    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw" event={"ID":"37f20e59-9d14-4441-b3ef-83a908e64e3b","Type":"ContainerStarted","Data":"756a8f5f1694c79a35c384378bf934b1ed069b81ac185d2fab4df683f7731a6d"}
Sep 10 11:53:19.779887 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:19.779851    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:19.879587 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:19.879537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:20.039025 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:20.038930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:20.151106 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:20.151037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:20.254836 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:20.254772    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:20.374395 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:20.374311    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm" event={"ID":"04d8a797-3778-4a8c-becf-401dd09db8d3","Type":"ContainerStarted","Data":"7409f0a458e9d8dc695ad7ed34caa428d87e552b1ce0efe2b980c5da5103cb81"}
Sep 10 11:53:20.397698 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:20.396537    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-nkzjm" podStartSLOduration=33.537411161 podStartE2EDuration="35.396517322s" totalImagesPullingTime="1.859106161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:20.395311385 +0000 UTC m=+110.998586297" watchObservedRunningTime="2026-09-10 11:53:20.396517322 +0000 UTC m=+110.999792234"
Sep 10 11:53:21.075723 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:21.075601    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:21.158122 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:21.158083    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:21.343956 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:21.343882    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f4dmp"
Sep 10 11:53:22.170995 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:22.170953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:22.271532 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:22.271497    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:23.328438 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:23.328408    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:23.387081 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:23.387050    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw" event={"ID":"37f20e59-9d14-4441-b3ef-83a908e64e3b","Type":"ContainerStarted","Data":"a0a9879f3f8cb3811b23f7b3287b65fc75e6246c16ac5bbb3fda2ed8bcef53bf"}
Sep 10 11:53:23.387253 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:23.387092    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw" event={"ID":"37f20e59-9d14-4441-b3ef-83a908e64e3b","Type":"ContainerStarted","Data":"ed395d91a1ce6d0967de7d9776cd039615875efdde4a3824c3964f9516918afa"}
Sep 10 11:53:23.387253 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:23.387111    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw" event={"ID":"37f20e59-9d14-4441-b3ef-83a908e64e3b","Type":"ContainerStarted","Data":"a57bdb13aa364aae32451abd23be66d1d7a3c533c51f40587f0d8ecbc3bc550b"}
Sep 10 11:53:23.406915 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:23.406866    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5895744555-cnznw" podStartSLOduration=2.345760846 podStartE2EDuration="6.406852185s" totalImagesPullingTime="4.061091339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:53:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:23.406197221 +0000 UTC m=+114.009472131" watchObservedRunningTime="2026-09-10 11:53:23.406852185 +0000 UTC m=+114.010127126"
Sep 10 11:53:23.410299 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:23.410271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:24.251613 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:24.251579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:24.361290 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:24.361264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:24.452972 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:24.452934    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:27.152423 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:27.152391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:27.155268 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:27.155231    2571 patch_prober.go:28] interesting pod/image-registry-cd6cfd846-t2kx6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 11:53:27.155381 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:27.155286    2571 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" podUID="fcf8bcfc-cb1d-4a39-960b-95070391d366" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 11:53:27.253487 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:27.253453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:27.370642 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:27.370610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.082663 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.082633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.188093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.188058    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.332432 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.332405    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:53:29.513048 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.513017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.632933 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.632896    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.756349 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.756311    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.929630 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.929605    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kn6d7"]
Sep 10 11:53:29.932796 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.932781    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.937792 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.937769    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 11:53:29.937903 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.937805    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 11:53:29.938026 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.938011    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 11:53:29.939084 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.938841    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:29.940177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.940125    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 11:53:29.942861 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.942837    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kl6rx\""
Sep 10 11:53:29.995001 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.994973    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-accelerators-collector-config\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-wtmp\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hddkn\" (UniqueName: \"kubernetes.io/projected/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-kube-api-access-hddkn\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995050    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-metrics-client-ca\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995101    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995147 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995120    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-root\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995179    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-textfile\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-tls\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:29.995335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:29.995227    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-sys\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.035446 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.035407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.095640 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095601    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-accelerators-collector-config\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095644    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-wtmp\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095672    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hddkn\" (UniqueName: \"kubernetes.io/projected/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-kube-api-access-hddkn\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095727    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-metrics-client-ca\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095755    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095786    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-root\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095805    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-wtmp\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095821    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-textfile\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095843    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-tls\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095873    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-root\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.095919    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-sys\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.096007    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-sys\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096085 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:53:30.096045    2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 11:53:30.096589 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:53:30.096132    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-tls podName:07bdaa2f-0f8e-416d-b5ab-c6d71d161c99 nodeName:}" failed. No retries permitted until 2026-09-10 11:53:30.596112244 +0000 UTC m=+121.199387139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-tls") pod "node-exporter-kn6d7" (UID: "07bdaa2f-0f8e-416d-b5ab-c6d71d161c99") : secret "node-exporter-tls" not found
Sep 10 11:53:30.096589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.096277    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-textfile\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.096295    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-accelerators-collector-config\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.096589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.096355    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-metrics-client-ca\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.098129 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.098112    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.106473 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.106433    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hddkn\" (UniqueName: \"kubernetes.io/projected/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-kube-api-access-hddkn\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.217976 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.217889    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.313983 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.313952    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.423783 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.423748    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.538013 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.537930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.601453 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.601416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-tls\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.603833 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.603806    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/07bdaa2f-0f8e-416d-b5ab-c6d71d161c99-node-exporter-tls\") pod \"node-exporter-kn6d7\" (UID: \"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99\") " pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.644916 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.644881    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.778428 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.778379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:30.845327 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.845261    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kn6d7"
Sep 10 11:53:30.856511 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:30.856481    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bdaa2f_0f8e_416d_b5ab_c6d71d161c99.slice/crio-22e0a77bcfe2c39936c5fbb8b3c68dfee008b04132aca5e70b7c8a784290790e WatchSource:0}: Error finding container 22e0a77bcfe2c39936c5fbb8b3c68dfee008b04132aca5e70b7c8a784290790e: Status 404 returned error can't find the container with id 22e0a77bcfe2c39936c5fbb8b3c68dfee008b04132aca5e70b7c8a784290790e
Sep 10 11:53:30.927955 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:30.927926    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:31.083161 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:31.083118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:31.414204 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:31.414142    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kn6d7" event={"ID":"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99","Type":"ContainerStarted","Data":"22e0a77bcfe2c39936c5fbb8b3c68dfee008b04132aca5e70b7c8a784290790e"}
Sep 10 11:53:31.524712 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:31.524680    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:31.630056 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:31.630010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:31.716312 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:31.716290    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:31.918351 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:31.918315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:32.418803 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:32.418772    2571 generic.go:356] "Generic (PLEG): container finished" podID="07bdaa2f-0f8e-416d-b5ab-c6d71d161c99" containerID="30d5e13e8b7b461237b8eb94c910e9e4a1026305bca535578757cd4a161d08ba" exitCode=0
Sep 10 11:53:32.419376 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:32.418856    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kn6d7" event={"ID":"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99","Type":"ContainerDied","Data":"30d5e13e8b7b461237b8eb94c910e9e4a1026305bca535578757cd4a161d08ba"}
Sep 10 11:53:32.689653 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:32.689611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:32.799805 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:32.799775    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:32.895331 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:32.895299    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:33.030240 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:33.030145    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:33.424779 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:33.424694    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kn6d7" event={"ID":"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99","Type":"ContainerStarted","Data":"8f691c4a8420615941f72ae4e78f6500c24c5290c42c773bc78e0b838cf77308"}
Sep 10 11:53:33.424779 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:33.424775    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kn6d7" event={"ID":"07bdaa2f-0f8e-416d-b5ab-c6d71d161c99","Type":"ContainerStarted","Data":"f3dd1714f59d3abf3fc66c92d58de8a80b6feabd0c31c869b9bd33a7931d9bc7"}
Sep 10 11:53:33.445462 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:33.445404    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kn6d7" podStartSLOduration=3.530255876 podStartE2EDuration="4.445388222s" totalImagesPullingTime="852.910079ms" totalInitContainerRuntime="62.222267ms" isStatefulPod=true podCreationTimestamp="2026-09-10 11:53:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:33.44457615 +0000 UTC m=+124.047851061" watchObservedRunningTime="2026-09-10 11:53:33.445388222 +0000 UTC m=+124.048663133"
Sep 10 11:53:34.281672 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.281635    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"]
Sep 10 11:53:34.284441 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.284426    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.288792 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.288764    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 11:53:34.288896 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.288807    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 11:53:34.288896 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.288860    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-c8fmf\""
Sep 10 11:53:34.288984 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.288949    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 11:53:34.289051 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.289032    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-a04hljjs0qgjk\""
Sep 10 11:53:34.289088 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.289051    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 11:53:34.302421 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.302396    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"]
Sep 10 11:53:34.433008 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.432973    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mrzw\" (UniqueName: \"kubernetes.io/projected/fff471a9-5e58-4d5c-92de-da6f253c51b0-kube-api-access-8mrzw\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.433408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.433013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fff471a9-5e58-4d5c-92de-da6f253c51b0-audit-log\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.433408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.433042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-secret-metrics-server-client-certs\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.433408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.433100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-secret-metrics-server-tls\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.433408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.433136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-client-ca-bundle\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.433408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.433185    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fff471a9-5e58-4d5c-92de-da6f253c51b0-metrics-server-audit-profiles\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.433408 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.433217    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fff471a9-5e58-4d5c-92de-da6f253c51b0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.434067 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.434046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:34.507758 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.507723    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:34.533960 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.533879    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fff471a9-5e58-4d5c-92de-da6f253c51b0-metrics-server-audit-profiles\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.533960 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.533924    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fff471a9-5e58-4d5c-92de-da6f253c51b0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534179 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534001    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mrzw\" (UniqueName: \"kubernetes.io/projected/fff471a9-5e58-4d5c-92de-da6f253c51b0-kube-api-access-8mrzw\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534179 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534032    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fff471a9-5e58-4d5c-92de-da6f253c51b0-audit-log\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534179 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534057    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-secret-metrics-server-client-certs\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534179 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-secret-metrics-server-tls\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534179 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534110    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-client-ca-bundle\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534803 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534775    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/fff471a9-5e58-4d5c-92de-da6f253c51b0-audit-log\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.534915 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534862    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fff471a9-5e58-4d5c-92de-da6f253c51b0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.535009 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.534988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/fff471a9-5e58-4d5c-92de-da6f253c51b0-metrics-server-audit-profiles\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.536763 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.536736    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-secret-metrics-server-tls\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.536866 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.536817    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-client-ca-bundle\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.536926 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.536879    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/fff471a9-5e58-4d5c-92de-da6f253c51b0-secret-metrics-server-client-certs\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.553454 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.553432    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mrzw\" (UniqueName: \"kubernetes.io/projected/fff471a9-5e58-4d5c-92de-da6f253c51b0-kube-api-access-8mrzw\") pod \"metrics-server-b7c6fc9cb-4b6tl\" (UID: \"fff471a9-5e58-4d5c-92de-da6f253c51b0\") " pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.593394 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.593362    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:34.729303 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.729274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:34.868552 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:34.868461    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"]
Sep 10 11:53:34.870884 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:34.870856    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff471a9_5e58_4d5c_92de_da6f253c51b0.slice/crio-b741af30e6d30201f30e4e1247b4c2f8133f5ed089b5f9951f42e51342c7c68c WatchSource:0}: Error finding container b741af30e6d30201f30e4e1247b4c2f8133f5ed089b5f9951f42e51342c7c68c: Status 404 returned error can't find the container with id b741af30e6d30201f30e4e1247b4c2f8133f5ed089b5f9951f42e51342c7c68c
Sep 10 11:53:35.281824 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.281792    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-567d5645c8-r7bml"]
Sep 10 11:53:35.286050 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.286033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.289409 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.289382    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 11:53:35.289409 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.289402    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lrm5q\""
Sep 10 11:53:35.289620 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.289470    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 11:53:35.290330 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.290302    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 11:53:35.290693 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.290675    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 11:53:35.290693 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.290687    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 11:53:35.290838 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.290768    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 11:53:35.292030 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.292012    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 11:53:35.295047 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.295027    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 11:53:35.298970 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.298949    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567d5645c8-r7bml"]
Sep 10 11:53:35.433447 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.433414    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl" event={"ID":"fff471a9-5e58-4d5c-92de-da6f253c51b0","Type":"ContainerStarted","Data":"b741af30e6d30201f30e4e1247b4c2f8133f5ed089b5f9951f42e51342c7c68c"}
Sep 10 11:53:35.440686 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440661    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-trusted-ca-bundle\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.440810 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440701    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-serving-cert\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.440810 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440719    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-console-config\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.440810 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440741    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-oauth-serving-cert\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.440926 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw7ns\" (UniqueName: \"kubernetes.io/projected/8b858185-a322-49bb-941e-a4bc5c18149b-kube-api-access-dw7ns\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.440926 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-service-ca\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.440926 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.440913    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-oauth-config\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541487 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-console-config\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541504    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-oauth-serving-cert\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7ns\" (UniqueName: \"kubernetes.io/projected/8b858185-a322-49bb-941e-a4bc5c18149b-kube-api-access-dw7ns\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541625    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-service-ca\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541798 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-oauth-config\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541798 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-trusted-ca-bundle\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.541798 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.541749    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-serving-cert\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.542282 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.542251    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-console-config\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.542786 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.542755    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-service-ca\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.543273 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.543251    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-oauth-serving-cert\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.543475 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.543448    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-trusted-ca-bundle\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.545572 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.545525    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-oauth-config\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.545670 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.545629    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-serving-cert\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.553840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.553818    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7ns\" (UniqueName: \"kubernetes.io/projected/8b858185-a322-49bb-941e-a4bc5c18149b-kube-api-access-dw7ns\") pod \"console-567d5645c8-r7bml\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") " pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.595782 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.595753    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:35.598432 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.598404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:35.702470 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.702436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:35.880453 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.880360    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-567d5645c8-r7bml"]
Sep 10 11:53:35.883919 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:35.883888    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b858185_a322_49bb_941e_a4bc5c18149b.slice/crio-9e21ca8cc66cf9f24c209ed1e66740a4f5cc58371e1c61a9707790f8d67d5af7 WatchSource:0}: Error finding container 9e21ca8cc66cf9f24c209ed1e66740a4f5cc58371e1c61a9707790f8d67d5af7: Status 404 returned error can't find the container with id 9e21ca8cc66cf9f24c209ed1e66740a4f5cc58371e1c61a9707790f8d67d5af7
Sep 10 11:53:35.980079 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:35.980035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:36.258915 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:36.258875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:36.356196 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:36.356120    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:36.440058 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:36.440023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl" event={"ID":"fff471a9-5e58-4d5c-92de-da6f253c51b0","Type":"ContainerStarted","Data":"3a3c86639fb1474e11c12d7fb7a9a73ad05a942c8b4e12feb1cd61eb02703be9"}
Sep 10 11:53:36.441351 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:36.441320    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d5645c8-r7bml" event={"ID":"8b858185-a322-49bb-941e-a4bc5c18149b","Type":"ContainerStarted","Data":"9e21ca8cc66cf9f24c209ed1e66740a4f5cc58371e1c61a9707790f8d67d5af7"}
Sep 10 11:53:36.464212 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:36.464137    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl" podStartSLOduration=0.988063553 podStartE2EDuration="2.464115627s" totalImagesPullingTime="1.476052074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:36.461768077 +0000 UTC m=+127.065043000" watchObservedRunningTime="2026-09-10 11:53:36.464115627 +0000 UTC m=+127.067390540"
Sep 10 11:53:39.068178 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.068133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:39.170421 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.170388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:39.453357 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.453323    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d5645c8-r7bml" event={"ID":"8b858185-a322-49bb-941e-a4bc5c18149b","Type":"ContainerStarted","Data":"de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f"}
Sep 10 11:53:39.478759 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.478713    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-567d5645c8-r7bml" podStartSLOduration=1.717415033 podStartE2EDuration="4.4786994s" totalImagesPullingTime="2.761284367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:39.478400716 +0000 UTC m=+130.081675626" watchObservedRunningTime="2026-09-10 11:53:39.4786994 +0000 UTC m=+130.081974379"
Sep 10 11:53:39.674590 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.674554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:53:39.676766 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.676746    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eb31aad-f7b6-456d-a8dd-717ea539a6e1-metrics-certs\") pod \"network-metrics-daemon-cfwkr\" (UID: \"6eb31aad-f7b6-456d-a8dd-717ea539a6e1\") " pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:53:39.868195 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.868101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:39.909161 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.909115    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zrvwr\""
Sep 10 11:53:39.917399 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.917378    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cfwkr"
Sep 10 11:53:39.932782 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:39.932758    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:40.105656 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.105625    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cd6cfd846-t2kx6"]
Sep 10 11:53:40.191712 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.191685    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cfwkr"]
Sep 10 11:53:40.194766 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:53:40.194742    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb31aad_f7b6_456d_a8dd_717ea539a6e1.slice/crio-fa7f7e2903bfc6e96e1cdd685c0d1604f261778e43c78b6aa5682d9821e5b082 WatchSource:0}: Error finding container fa7f7e2903bfc6e96e1cdd685c0d1604f261778e43c78b6aa5682d9821e5b082: Status 404 returned error can't find the container with id fa7f7e2903bfc6e96e1cdd685c0d1604f261778e43c78b6aa5682d9821e5b082
Sep 10 11:53:40.225281 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.225239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:40.307029 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.307000    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:40.458060 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.457974    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cfwkr" event={"ID":"6eb31aad-f7b6-456d-a8dd-717ea539a6e1","Type":"ContainerStarted","Data":"fa7f7e2903bfc6e96e1cdd685c0d1604f261778e43c78b6aa5682d9821e5b082"}
Sep 10 11:53:40.477809 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.477783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:40.579356 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:40.579324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:42.466078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:42.466041    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cfwkr" event={"ID":"6eb31aad-f7b6-456d-a8dd-717ea539a6e1","Type":"ContainerStarted","Data":"b6b694cd72a39799c1aa975ded09ca171438f31c85c76e80e559df06d49f3a45"}
Sep 10 11:53:42.466474 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:42.466096    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cfwkr" event={"ID":"6eb31aad-f7b6-456d-a8dd-717ea539a6e1","Type":"ContainerStarted","Data":"e10b2f292307572fb9a2ab5764b7c5f3fd9f55c6e5a7d01320c7db4c0b51c06c"}
Sep 10 11:53:42.483452 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:42.483404    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cfwkr" podStartSLOduration=131.22702016 podStartE2EDuration="2m12.48338782s" totalImagesPullingTime="1.25636766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:51:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:53:42.482410104 +0000 UTC m=+133.085685016" watchObservedRunningTime="2026-09-10 11:53:42.48338782 +0000 UTC m=+133.086662771"
Sep 10 11:53:42.594304 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:42.594269    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:42.658432 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:42.658398    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:44.315312 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:44.315275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:44.411693 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:44.411649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:45.596610 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:45.596574    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:45.597005 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:45.596634    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:45.601340 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:45.601318    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:46.487640 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:46.487606    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:53:46.662119 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:46.662088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:46.763651 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:46.763583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:46.830838 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:46.830812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:48.536286 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:48.536252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:48.637816 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:48.637774    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:49.144303 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:49.144273    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:49.243804 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:49.243778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:50.716444 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:50.716410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:50.821115 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:50.821079    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:54.045079 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:54.045042    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:54.115337 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:54.115309    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:53:54.593946 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:54.593915    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:53:54.594259 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:53:54.593969    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:54:05.124782 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.124742    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" podUID="fcf8bcfc-cb1d-4a39-960b-95070391d366" containerName="registry" containerID="cri-o://0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc" gracePeriod=30
Sep 10 11:54:05.363327 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.363304    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:54:05.468539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468512    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-bound-sa-token\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.468707 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468547    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.468707 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468568    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-installation-pull-secrets\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.468707 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468601    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-image-registry-private-configuration\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.468707 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468649    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-trusted-ca\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.468707 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468674    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx4s9\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-kube-api-access-lx4s9\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.468978 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.468717    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-certificates\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.469175 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.469103    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:54:05.469300 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.469181    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fcf8bcfc-cb1d-4a39-960b-95070391d366-ca-trust-extracted\") pod \"fcf8bcfc-cb1d-4a39-960b-95070391d366\" (UID: \"fcf8bcfc-cb1d-4a39-960b-95070391d366\") "
Sep 10 11:54:05.469482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.469424    2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-certificates\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.469482 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.469149    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:54:05.470997 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.470967    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:54:05.471116 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.471091    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-kube-api-access-lx4s9" (OuterVolumeSpecName: "kube-api-access-lx4s9") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "kube-api-access-lx4s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:54:05.471243 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.471216    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:54:05.471348 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.471254    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:54:05.471392 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.471345    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:54:05.477623 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.477599    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf8bcfc-cb1d-4a39-960b-95070391d366-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fcf8bcfc-cb1d-4a39-960b-95070391d366" (UID: "fcf8bcfc-cb1d-4a39-960b-95070391d366"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:54:05.549444 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.549419    2571 generic.go:356] "Generic (PLEG): container finished" podID="fcf8bcfc-cb1d-4a39-960b-95070391d366" containerID="0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc" exitCode=0
Sep 10 11:54:05.549546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.549458    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6"
Sep 10 11:54:05.549598 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.549457    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" event={"ID":"fcf8bcfc-cb1d-4a39-960b-95070391d366","Type":"ContainerDied","Data":"0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc"}
Sep 10 11:54:05.549665 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.549612    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-cd6cfd846-t2kx6" event={"ID":"fcf8bcfc-cb1d-4a39-960b-95070391d366","Type":"ContainerDied","Data":"185b36b55e8bdf7c425e8a5c1c256deb9e8f701450ac43d0ae791ca3ed6e1b2a"}
Sep 10 11:54:05.549717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.549667    2571 scope.go:117] "RemoveContainer" containerID="0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc"
Sep 10 11:54:05.557471 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.557452    2571 scope.go:117] "RemoveContainer" containerID="0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc"
Sep 10 11:54:05.557740 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:54:05.557716    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc\": container with ID starting with 0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc not found: ID does not exist" containerID="0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc"
Sep 10 11:54:05.557797 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.557748    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc"} err="failed to get container status \"0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc\": rpc error: code = NotFound desc = could not find container \"0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc\": container with ID starting with 0fb9d62ee464979508173fb7901a4c40d43ca55934b6bfc76db6a13b492364fc not found: ID does not exist"
Sep 10 11:54:05.569052 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.569032    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-cd6cfd846-t2kx6"]
Sep 10 11:54:05.570098 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570077    2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fcf8bcfc-cb1d-4a39-960b-95070391d366-ca-trust-extracted\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.570177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570100    2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-bound-sa-token\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.570177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570110    2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-registry-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.570177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570118    2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-installation-pull-secrets\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.570177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570128    2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fcf8bcfc-cb1d-4a39-960b-95070391d366-image-registry-private-configuration\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.570177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570136    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fcf8bcfc-cb1d-4a39-960b-95070391d366-trusted-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.570177 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.570144    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx4s9\" (UniqueName: \"kubernetes.io/projected/fcf8bcfc-cb1d-4a39-960b-95070391d366-kube-api-access-lx4s9\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:54:05.574097 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.574076    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-cd6cfd846-t2kx6"]
Sep 10 11:54:05.893830 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:05.893757    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf8bcfc-cb1d-4a39-960b-95070391d366" path="/var/lib/kubelet/pods/fcf8bcfc-cb1d-4a39-960b-95070391d366/volumes"
Sep 10 11:54:09.052427 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:09.052384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:09.152062 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:09.152029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:09.565308 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:09.565277    2571 generic.go:356] "Generic (PLEG): container finished" podID="ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab" containerID="677beca8803b35fb1145ded4145a274c7623ab4d592ab4d49b254f4416fce18a" exitCode=0
Sep 10 11:54:09.565506 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:09.565352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m" event={"ID":"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab","Type":"ContainerDied","Data":"677beca8803b35fb1145ded4145a274c7623ab4d592ab4d49b254f4416fce18a"}
Sep 10 11:54:09.565675 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:09.565660    2571 scope.go:117] "RemoveContainer" containerID="677beca8803b35fb1145ded4145a274c7623ab4d592ab4d49b254f4416fce18a"
Sep 10 11:54:10.570960 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:10.570926    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-4mt7m" event={"ID":"ecf4b6cd-c1fb-40b4-a44a-af1ab7ed12ab","Type":"ContainerStarted","Data":"1658f3ca1ae4fbb85138451916462ef702121c3e0f95c0277e0fce61db1b89fb"}
Sep 10 11:54:14.599418 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:14.599393    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:54:14.603057 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:14.603033    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b7c6fc9cb-4b6tl"
Sep 10 11:54:44.897494 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:44.897463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:49.045763 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:49.045726    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:49.270877 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:49.270848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:57.109628 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:57.109581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:58.577292 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.577261    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c68db944-b8wb9"]
Sep 10 11:54:58.577715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.577508    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf8bcfc-cb1d-4a39-960b-95070391d366" containerName="registry"
Sep 10 11:54:58.577715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.577519    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf8bcfc-cb1d-4a39-960b-95070391d366" containerName="registry"
Sep 10 11:54:58.577715 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.577573    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf8bcfc-cb1d-4a39-960b-95070391d366" containerName="registry"
Sep 10 11:54:58.580358 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.580341    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.592343 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.592319    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c68db944-b8wb9"]
Sep 10 11:54:58.621086 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-oauth-config\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.621241 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621092    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxw75\" (UniqueName: \"kubernetes.io/projected/70362c98-438d-40b5-a1c1-82b65e753ebb-kube-api-access-bxw75\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.621241 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-trusted-ca-bundle\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.621241 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-serving-cert\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.621241 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621238    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-oauth-serving-cert\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.621398 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621260    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-console-config\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.621398 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.621283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-service-ca\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.634612 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.634592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:54:58.722641 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722607    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-trusted-ca-bundle\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.722813 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-serving-cert\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.722813 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722687    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-oauth-serving-cert\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.722813 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722709    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-console-config\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.722813 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722732    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-service-ca\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.722813 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722758    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-oauth-config\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.723072 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.722895    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxw75\" (UniqueName: \"kubernetes.io/projected/70362c98-438d-40b5-a1c1-82b65e753ebb-kube-api-access-bxw75\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.723558 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.723534    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-oauth-serving-cert\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.723657 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.723608    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-service-ca\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.723657 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.723608    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-console-config\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.723728 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.723690    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-trusted-ca-bundle\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.725221 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.725195    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-oauth-config\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.725443 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.725419    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-serving-cert\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.731911 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.731891    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxw75\" (UniqueName: \"kubernetes.io/projected/70362c98-438d-40b5-a1c1-82b65e753ebb-kube-api-access-bxw75\") pod \"console-78c68db944-b8wb9\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") " pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:58.888836 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:58.888749    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:54:59.162843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:59.162815    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c68db944-b8wb9"]
Sep 10 11:54:59.165259 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:54:59.165233    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70362c98_438d_40b5_a1c1_82b65e753ebb.slice/crio-dc2fa1f4422afe5a323553df7b1fa87acdf12c27b9328199df52e0308fb3ea6d WatchSource:0}: Error finding container dc2fa1f4422afe5a323553df7b1fa87acdf12c27b9328199df52e0308fb3ea6d: Status 404 returned error can't find the container with id dc2fa1f4422afe5a323553df7b1fa87acdf12c27b9328199df52e0308fb3ea6d
Sep 10 11:54:59.738323 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:59.738287    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c68db944-b8wb9" event={"ID":"70362c98-438d-40b5-a1c1-82b65e753ebb","Type":"ContainerStarted","Data":"c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1"}
Sep 10 11:54:59.738712 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:59.738365    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c68db944-b8wb9" event={"ID":"70362c98-438d-40b5-a1c1-82b65e753ebb","Type":"ContainerStarted","Data":"dc2fa1f4422afe5a323553df7b1fa87acdf12c27b9328199df52e0308fb3ea6d"}
Sep 10 11:54:59.759239 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:54:59.759188    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78c68db944-b8wb9" podStartSLOduration=1.759148344 podStartE2EDuration="1.759148344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:54:59.75840335 +0000 UTC m=+210.361678262" watchObservedRunningTime="2026-09-10 11:54:59.759148344 +0000 UTC m=+210.362423268"
Sep 10 11:55:04.281082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:04.281044    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:04.402313 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:04.402276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:07.331422 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:07.331388    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c68db944-b8wb9"]
Sep 10 11:55:08.889030 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:08.888996    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:55:16.220785 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:16.220754    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:16.295738 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:16.295711    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:18.594144 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:18.594109    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-567d5645c8-r7bml"]
Sep 10 11:55:18.733967 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:18.733912    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:18.835503 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:18.835463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:18.901231 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:18.901130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:24.061521 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:24.061486    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:24.163291 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:24.163264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:24.232753 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:24.232707    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:32.350912 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.350836    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78c68db944-b8wb9" podUID="70362c98-438d-40b5-a1c1-82b65e753ebb" containerName="console" containerID="cri-o://c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1" gracePeriod=15
Sep 10 11:55:32.582078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.582052    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:55:32.669317 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-trusted-ca-bundle\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669317 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669271    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxw75\" (UniqueName: \"kubernetes.io/projected/70362c98-438d-40b5-a1c1-82b65e753ebb-kube-api-access-bxw75\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669319    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-console-config\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669354    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-oauth-serving-cert\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669380    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-serving-cert\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669401    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-oauth-config\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669423    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-service-ca\") pod \"70362c98-438d-40b5-a1c1-82b65e753ebb\" (UID: \"70362c98-438d-40b5-a1c1-82b65e753ebb\") "
Sep 10 11:55:32.669864 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669836    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:32.669924 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669894    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:32.669960 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.669927    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-console-config" (OuterVolumeSpecName: "console-config") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:32.670049 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.670024    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-service-ca" (OuterVolumeSpecName: "service-ca") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:32.671681 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.671650    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70362c98-438d-40b5-a1c1-82b65e753ebb-kube-api-access-bxw75" (OuterVolumeSpecName: "kube-api-access-bxw75") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "kube-api-access-bxw75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:55:32.671779 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.671675    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:55:32.671779 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.671705    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70362c98-438d-40b5-a1c1-82b65e753ebb" (UID: "70362c98-438d-40b5-a1c1-82b65e753ebb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:55:32.770056 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770019    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-serving-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.770056 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770046    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70362c98-438d-40b5-a1c1-82b65e753ebb-console-oauth-config\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.770056 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770057    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.770056 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770065    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-trusted-ca-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.770328 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770074    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxw75\" (UniqueName: \"kubernetes.io/projected/70362c98-438d-40b5-a1c1-82b65e753ebb-kube-api-access-bxw75\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.770328 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770082    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-console-config\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.770328 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.770090    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70362c98-438d-40b5-a1c1-82b65e753ebb-oauth-serving-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:32.848266 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.848226    2571 generic.go:356] "Generic (PLEG): container finished" podID="70362c98-438d-40b5-a1c1-82b65e753ebb" containerID="c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1" exitCode=0
Sep 10 11:55:32.848454 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.848277    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c68db944-b8wb9"
Sep 10 11:55:32.848454 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.848320    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c68db944-b8wb9" event={"ID":"70362c98-438d-40b5-a1c1-82b65e753ebb","Type":"ContainerDied","Data":"c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1"}
Sep 10 11:55:32.848454 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.848386    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c68db944-b8wb9" event={"ID":"70362c98-438d-40b5-a1c1-82b65e753ebb","Type":"ContainerDied","Data":"dc2fa1f4422afe5a323553df7b1fa87acdf12c27b9328199df52e0308fb3ea6d"}
Sep 10 11:55:32.848454 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.848415    2571 scope.go:117] "RemoveContainer" containerID="c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1"
Sep 10 11:55:32.857274 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.857255    2571 scope.go:117] "RemoveContainer" containerID="c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1"
Sep 10 11:55:32.857543 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:55:32.857526    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1\": container with ID starting with c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1 not found: ID does not exist" containerID="c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1"
Sep 10 11:55:32.857590 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.857553    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1"} err="failed to get container status \"c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1\": rpc error: code = NotFound desc = could not find container \"c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1\": container with ID starting with c4951f3c73c556697e23c8b042735f7186158ede6aa09b3e16b304eb4a56ddf1 not found: ID does not exist"
Sep 10 11:55:32.870069 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.870042    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78c68db944-b8wb9"]
Sep 10 11:55:32.878275 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:32.878246    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78c68db944-b8wb9"]
Sep 10 11:55:33.891350 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:33.891316    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70362c98-438d-40b5-a1c1-82b65e753ebb" path="/var/lib/kubelet/pods/70362c98-438d-40b5-a1c1-82b65e753ebb/volumes"
Sep 10 11:55:43.614727 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.614691    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-567d5645c8-r7bml" podUID="8b858185-a322-49bb-941e-a4bc5c18149b" containerName="console" containerID="cri-o://de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f" gracePeriod=15
Sep 10 11:55:43.856614 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.856586    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:55:43.888122 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.888049    2571 generic.go:356] "Generic (PLEG): container finished" podID="8b858185-a322-49bb-941e-a4bc5c18149b" containerID="de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f" exitCode=0
Sep 10 11:55:43.888122 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.888112    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-567d5645c8-r7bml"
Sep 10 11:55:43.892302 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.892275    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d5645c8-r7bml" event={"ID":"8b858185-a322-49bb-941e-a4bc5c18149b","Type":"ContainerDied","Data":"de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f"}
Sep 10 11:55:43.892431 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.892325    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-567d5645c8-r7bml" event={"ID":"8b858185-a322-49bb-941e-a4bc5c18149b","Type":"ContainerDied","Data":"9e21ca8cc66cf9f24c209ed1e66740a4f5cc58371e1c61a9707790f8d67d5af7"}
Sep 10 11:55:43.892431 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.892353    2571 scope.go:117] "RemoveContainer" containerID="de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f"
Sep 10 11:55:43.899688 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.899671    2571 scope.go:117] "RemoveContainer" containerID="de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f"
Sep 10 11:55:43.899969 ip-10-0-129-41 kubenswrapper[2571]: E0910 11:55:43.899939    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f\": container with ID starting with de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f not found: ID does not exist" containerID="de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f"
Sep 10 11:55:43.900017 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.899967    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f"} err="failed to get container status \"de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f\": rpc error: code = NotFound desc = could not find container \"de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f\": container with ID starting with de289932c2bf333011f8e91c73e525ab8e04a66425cda0a58c9fb1876c831d3f not found: ID does not exist"
Sep 10 11:55:43.951253 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951231    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw7ns\" (UniqueName: \"kubernetes.io/projected/8b858185-a322-49bb-941e-a4bc5c18149b-kube-api-access-dw7ns\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951347 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951276    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-trusted-ca-bundle\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951347 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951296    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-oauth-serving-cert\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951347 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951314    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-serving-cert\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951347 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951338    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-oauth-config\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951516 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951356    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-console-config\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951516 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951404    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-service-ca\") pod \"8b858185-a322-49bb-941e-a4bc5c18149b\" (UID: \"8b858185-a322-49bb-941e-a4bc5c18149b\") "
Sep 10 11:55:43.951778 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951750    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:43.951865 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951838    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-service-ca" (OuterVolumeSpecName: "service-ca") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:43.951932 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951804    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-console-config" (OuterVolumeSpecName: "console-config") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:43.951987 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.951934    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:55:43.953432 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.953402    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:55:43.953432 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.953415    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b858185-a322-49bb-941e-a4bc5c18149b-kube-api-access-dw7ns" (OuterVolumeSpecName: "kube-api-access-dw7ns") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "kube-api-access-dw7ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:55:43.953551 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:43.953414    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8b858185-a322-49bb-941e-a4bc5c18149b" (UID: "8b858185-a322-49bb-941e-a4bc5c18149b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:55:44.052013 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.051984    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.052013 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.052012    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw7ns\" (UniqueName: \"kubernetes.io/projected/8b858185-a322-49bb-941e-a4bc5c18149b-kube-api-access-dw7ns\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.052210 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.052024    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-trusted-ca-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.052210 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.052038    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-oauth-serving-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.052210 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.052056    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-serving-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.052210 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.052068    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b858185-a322-49bb-941e-a4bc5c18149b-console-oauth-config\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.052210 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.052081    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b858185-a322-49bb-941e-a4bc5c18149b-console-config\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:55:44.209717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.209690    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-567d5645c8-r7bml"]
Sep 10 11:55:44.214842 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:44.214810    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-567d5645c8-r7bml"]
Sep 10 11:55:45.891529 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:45.891499    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b858185-a322-49bb-941e-a4bc5c18149b" path="/var/lib/kubelet/pods/8b858185-a322-49bb-941e-a4bc5c18149b/volumes"
Sep 10 11:55:46.280930 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:46.280898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:55:46.391864 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:55:46.391832    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:05.984191 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:05.984135    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:06.590029 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:06.589990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:06.692630 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:06.692602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:07.626614 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.626573    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"]
Sep 10 11:56:07.627078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.626927    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70362c98-438d-40b5-a1c1-82b65e753ebb" containerName="console"
Sep 10 11:56:07.627078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.626944    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70362c98-438d-40b5-a1c1-82b65e753ebb" containerName="console"
Sep 10 11:56:07.627078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.626962    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b858185-a322-49bb-941e-a4bc5c18149b" containerName="console"
Sep 10 11:56:07.627078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.626969    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b858185-a322-49bb-941e-a4bc5c18149b" containerName="console"
Sep 10 11:56:07.627078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.627032    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70362c98-438d-40b5-a1c1-82b65e753ebb" containerName="console"
Sep 10 11:56:07.627078 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.627043    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b858185-a322-49bb-941e-a4bc5c18149b" containerName="console"
Sep 10 11:56:07.629822 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.629800    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.632377 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.632353    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 11:56:07.632476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.632376    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 11:56:07.633168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.633142    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8dsfc\""
Sep 10 11:56:07.643799 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.643778    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"]
Sep 10 11:56:07.714099 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.714068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rksbk\" (UniqueName: \"kubernetes.io/projected/3ecb21d0-7a53-402d-bf42-b58fa13ac372-kube-api-access-rksbk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.714290 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.714106    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.714290 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.714127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.814490 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.814458    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rksbk\" (UniqueName: \"kubernetes.io/projected/3ecb21d0-7a53-402d-bf42-b58fa13ac372-kube-api-access-rksbk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.814660 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.814496    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.814660 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.814515    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.814849 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.814829    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.814908 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.814857    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.823508 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.823476    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rksbk\" (UniqueName: \"kubernetes.io/projected/3ecb21d0-7a53-402d-bf42-b58fa13ac372-kube-api-access-rksbk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:07.939183 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:07.939144    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:08.214474 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:08.214401    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"]
Sep 10 11:56:08.217846 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:56:08.217814    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecb21d0_7a53_402d_bf42_b58fa13ac372.slice/crio-80d218c53f03b2abd1f216e4c36810605a8fa00303cdc8a23dfd4fde2bee63b6 WatchSource:0}: Error finding container 80d218c53f03b2abd1f216e4c36810605a8fa00303cdc8a23dfd4fde2bee63b6: Status 404 returned error can't find the container with id 80d218c53f03b2abd1f216e4c36810605a8fa00303cdc8a23dfd4fde2bee63b6
Sep 10 11:56:08.977304 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:08.977264    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6" event={"ID":"3ecb21d0-7a53-402d-bf42-b58fa13ac372","Type":"ContainerStarted","Data":"80d218c53f03b2abd1f216e4c36810605a8fa00303cdc8a23dfd4fde2bee63b6"}
Sep 10 11:56:15.001263 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:15.001231    2571 generic.go:356] "Generic (PLEG): container finished" podID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerID="74498c44d1b0cce7f3c4af00eaa4f0fe3d3dc04515ed1a097a91c6592ccb1d21" exitCode=0
Sep 10 11:56:15.001685 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:15.001320    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6" event={"ID":"3ecb21d0-7a53-402d-bf42-b58fa13ac372","Type":"ContainerDied","Data":"74498c44d1b0cce7f3c4af00eaa4f0fe3d3dc04515ed1a097a91c6592ccb1d21"}
Sep 10 11:56:18.014598 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:18.014556    2571 generic.go:356] "Generic (PLEG): container finished" podID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerID="df336ce44b64d2aecb4b5c982298354d9f7aebcf28d0dca889f9ef578dd13881" exitCode=0
Sep 10 11:56:18.014998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:18.014638    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6" event={"ID":"3ecb21d0-7a53-402d-bf42-b58fa13ac372","Type":"ContainerDied","Data":"df336ce44b64d2aecb4b5c982298354d9f7aebcf28d0dca889f9ef578dd13881"}
Sep 10 11:56:25.044527 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:25.044496    2571 generic.go:356] "Generic (PLEG): container finished" podID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerID="56c04167c8819704aa749cbba6dff7b00dbbdcabcb46f8f03cf875e30d519550" exitCode=0
Sep 10 11:56:25.044955 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:25.044583    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6" event={"ID":"3ecb21d0-7a53-402d-bf42-b58fa13ac372","Type":"ContainerDied","Data":"56c04167c8819704aa749cbba6dff7b00dbbdcabcb46f8f03cf875e30d519550"}
Sep 10 11:56:26.166344 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.166322    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:26.257165 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.257134    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-util\") pod \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") "
Sep 10 11:56:26.257293 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.257195    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rksbk\" (UniqueName: \"kubernetes.io/projected/3ecb21d0-7a53-402d-bf42-b58fa13ac372-kube-api-access-rksbk\") pod \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") "
Sep 10 11:56:26.257293 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.257225    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-bundle\") pod \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\" (UID: \"3ecb21d0-7a53-402d-bf42-b58fa13ac372\") "
Sep 10 11:56:26.257855 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.257829    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-bundle" (OuterVolumeSpecName: "bundle") pod "3ecb21d0-7a53-402d-bf42-b58fa13ac372" (UID: "3ecb21d0-7a53-402d-bf42-b58fa13ac372"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:56:26.259305 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.259277    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ecb21d0-7a53-402d-bf42-b58fa13ac372-kube-api-access-rksbk" (OuterVolumeSpecName: "kube-api-access-rksbk") pod "3ecb21d0-7a53-402d-bf42-b58fa13ac372" (UID: "3ecb21d0-7a53-402d-bf42-b58fa13ac372"). InnerVolumeSpecName "kube-api-access-rksbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:56:26.261301 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.261274    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-util" (OuterVolumeSpecName: "util") pod "3ecb21d0-7a53-402d-bf42-b58fa13ac372" (UID: "3ecb21d0-7a53-402d-bf42-b58fa13ac372"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:56:26.357801 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.357730    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:56:26.357801 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.357756    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rksbk\" (UniqueName: \"kubernetes.io/projected/3ecb21d0-7a53-402d-bf42-b58fa13ac372-kube-api-access-rksbk\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:56:26.357801 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:26.357767    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ecb21d0-7a53-402d-bf42-b58fa13ac372-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:56:27.053301 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:27.053265    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6" event={"ID":"3ecb21d0-7a53-402d-bf42-b58fa13ac372","Type":"ContainerDied","Data":"80d218c53f03b2abd1f216e4c36810605a8fa00303cdc8a23dfd4fde2bee63b6"}
Sep 10 11:56:27.053461 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:27.053311    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80d218c53f03b2abd1f216e4c36810605a8fa00303cdc8a23dfd4fde2bee63b6"
Sep 10 11:56:27.053461 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:27.053316    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540h86k6"
Sep 10 11:56:28.876379 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:28.876344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:29.841000 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:29.840971    2571 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 11:56:30.229164 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.229119    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:30.827733 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827700    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"]
Sep 10 11:56:30.827934 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827922    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="util"
Sep 10 11:56:30.827934 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827935    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="util"
Sep 10 11:56:30.828028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827942    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="pull"
Sep 10 11:56:30.828028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827948    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="pull"
Sep 10 11:56:30.828028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827954    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="extract"
Sep 10 11:56:30.828028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.827960    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="extract"
Sep 10 11:56:30.828028 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.828012    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ecb21d0-7a53-402d-bf42-b58fa13ac372" containerName="extract"
Sep 10 11:56:30.830577 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.830560    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:30.834787 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.833662    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 11:56:30.834787 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.833693    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-bbdtn\""
Sep 10 11:56:30.834787 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.834212    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:56:30.846172 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.846133    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"]
Sep 10 11:56:30.887309 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.887285    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7965eb2-2e9d-4745-b986-6d5c05ae70dd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s9g56\" (UID: \"b7965eb2-2e9d-4745-b986-6d5c05ae70dd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:30.887421 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.887318    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhc8x\" (UniqueName: \"kubernetes.io/projected/b7965eb2-2e9d-4745-b986-6d5c05ae70dd-kube-api-access-nhc8x\") pod \"cert-manager-operator-controller-manager-55c444bf86-s9g56\" (UID: \"b7965eb2-2e9d-4745-b986-6d5c05ae70dd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:30.988613 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.988582    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7965eb2-2e9d-4745-b986-6d5c05ae70dd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s9g56\" (UID: \"b7965eb2-2e9d-4745-b986-6d5c05ae70dd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:30.988731 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.988623    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhc8x\" (UniqueName: \"kubernetes.io/projected/b7965eb2-2e9d-4745-b986-6d5c05ae70dd-kube-api-access-nhc8x\") pod \"cert-manager-operator-controller-manager-55c444bf86-s9g56\" (UID: \"b7965eb2-2e9d-4745-b986-6d5c05ae70dd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:30.989012 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.988994    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7965eb2-2e9d-4745-b986-6d5c05ae70dd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s9g56\" (UID: \"b7965eb2-2e9d-4745-b986-6d5c05ae70dd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:30.999407 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:30.999377    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhc8x\" (UniqueName: \"kubernetes.io/projected/b7965eb2-2e9d-4745-b986-6d5c05ae70dd-kube-api-access-nhc8x\") pod \"cert-manager-operator-controller-manager-55c444bf86-s9g56\" (UID: \"b7965eb2-2e9d-4745-b986-6d5c05ae70dd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:31.141568 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:31.141500    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"
Sep 10 11:56:31.429926 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:31.429894    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56"]
Sep 10 11:56:31.434633 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:56:31.434595    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7965eb2_2e9d_4745_b986_6d5c05ae70dd.slice/crio-eff3c4f04f465440d5d4e5a4f78ce4283fef7f75d97e0b2b11a62162e4e6c7ac WatchSource:0}: Error finding container eff3c4f04f465440d5d4e5a4f78ce4283fef7f75d97e0b2b11a62162e4e6c7ac: Status 404 returned error can't find the container with id eff3c4f04f465440d5d4e5a4f78ce4283fef7f75d97e0b2b11a62162e4e6c7ac
Sep 10 11:56:31.437390 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:31.437370    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:56:32.071744 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:32.071712    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56" event={"ID":"b7965eb2-2e9d-4745-b986-6d5c05ae70dd","Type":"ContainerStarted","Data":"eff3c4f04f465440d5d4e5a4f78ce4283fef7f75d97e0b2b11a62162e4e6c7ac"}
Sep 10 11:56:32.493641 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:32.493609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:32.593476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:32.593448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:33.466757 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:33.466728    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:34.081698 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:34.081660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56" event={"ID":"b7965eb2-2e9d-4745-b986-6d5c05ae70dd","Type":"ContainerStarted","Data":"8cd3a4d3643c235cdcd1b9049d74ed6974541a3fb5c6146f237537ff1929be23"}
Sep 10 11:56:34.109528 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:34.109473    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s9g56" podStartSLOduration=1.60104321 podStartE2EDuration="4.109456924s" totalImagesPullingTime="2.508413714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:56:34.108634179 +0000 UTC m=+304.711909090" watchObservedRunningTime="2026-09-10 11:56:34.109456924 +0000 UTC m=+304.712731835"
Sep 10 11:56:34.545752 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:34.545712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:34.692488 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:34.692444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:34.987786 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:34.987752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:35.256863 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.256787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:35.361001 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.360968    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"]
Sep 10 11:56:35.386079 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.386054    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"]
Sep 10 11:56:35.386267 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.386181    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.388945 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.388919    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8dsfc\""
Sep 10 11:56:35.389065 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.388964    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 11:56:35.389608 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.389589    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 11:56:35.421217 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.421190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.421322 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.421242    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.421322 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.421273    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwb4c\" (UniqueName: \"kubernetes.io/projected/10cf5c11-aa9a-4069-abc3-31aee27819d3-kube-api-access-pwb4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.522422 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.522368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.522422 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.522402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.522545 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.522422    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwb4c\" (UniqueName: \"kubernetes.io/projected/10cf5c11-aa9a-4069-abc3-31aee27819d3-kube-api-access-pwb4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.522743 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.522724    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.522782 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.522757    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.532009 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.531971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwb4c\" (UniqueName: \"kubernetes.io/projected/10cf5c11-aa9a-4069-abc3-31aee27819d3-kube-api-access-pwb4c\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.695385 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.695351    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:35.985286 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:35.985259    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"]
Sep 10 11:56:35.988135 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:56:35.988109    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10cf5c11_aa9a_4069_abc3_31aee27819d3.slice/crio-6f5a2320b88f9a941cc03f369069affb7a9947def3e62c979cee0ce29a4fdb42 WatchSource:0}: Error finding container 6f5a2320b88f9a941cc03f369069affb7a9947def3e62c979cee0ce29a4fdb42: Status 404 returned error can't find the container with id 6f5a2320b88f9a941cc03f369069affb7a9947def3e62c979cee0ce29a4fdb42
Sep 10 11:56:36.090957 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:36.090930    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr" event={"ID":"10cf5c11-aa9a-4069-abc3-31aee27819d3","Type":"ContainerStarted","Data":"2ac97a46fa1749b12e1e6e9379e48918e6842d437ad14aa65865bc664e20a172"}
Sep 10 11:56:36.091048 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:36.090999    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr" event={"ID":"10cf5c11-aa9a-4069-abc3-31aee27819d3","Type":"ContainerStarted","Data":"6f5a2320b88f9a941cc03f369069affb7a9947def3e62c979cee0ce29a4fdb42"}
Sep 10 11:56:37.096961 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.096929    2571 generic.go:356] "Generic (PLEG): container finished" podID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerID="2ac97a46fa1749b12e1e6e9379e48918e6842d437ad14aa65865bc664e20a172" exitCode=0
Sep 10 11:56:37.097343 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.096990    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr" event={"ID":"10cf5c11-aa9a-4069-abc3-31aee27819d3","Type":"ContainerDied","Data":"2ac97a46fa1749b12e1e6e9379e48918e6842d437ad14aa65865bc664e20a172"}
Sep 10 11:56:37.329820 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.329785    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:37.715103 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.715064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:37.738327 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.738299    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"]
Sep 10 11:56:37.752803 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.752781    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:37.753364 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.753343    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"]
Sep 10 11:56:37.755363 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.755341    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-28cnw\""
Sep 10 11:56:37.756093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.756072    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 11:56:37.756216 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.756096    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 11:56:37.842130 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.842100    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ca049a5-b8f1-4c78-9a71-2daa88462959-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8wjlh\" (UID: \"7ca049a5-b8f1-4c78-9a71-2daa88462959\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:37.842294 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.842135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b2vw\" (UniqueName: \"kubernetes.io/projected/7ca049a5-b8f1-4c78-9a71-2daa88462959-kube-api-access-6b2vw\") pod \"cert-manager-cainjector-559cdd8c87-8wjlh\" (UID: \"7ca049a5-b8f1-4c78-9a71-2daa88462959\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:37.942851 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.942819    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ca049a5-b8f1-4c78-9a71-2daa88462959-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8wjlh\" (UID: \"7ca049a5-b8f1-4c78-9a71-2daa88462959\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:37.942992 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.942857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b2vw\" (UniqueName: \"kubernetes.io/projected/7ca049a5-b8f1-4c78-9a71-2daa88462959-kube-api-access-6b2vw\") pod \"cert-manager-cainjector-559cdd8c87-8wjlh\" (UID: \"7ca049a5-b8f1-4c78-9a71-2daa88462959\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:37.956296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.956257    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b2vw\" (UniqueName: \"kubernetes.io/projected/7ca049a5-b8f1-4c78-9a71-2daa88462959-kube-api-access-6b2vw\") pod \"cert-manager-cainjector-559cdd8c87-8wjlh\" (UID: \"7ca049a5-b8f1-4c78-9a71-2daa88462959\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:37.956427 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:37.956405    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7ca049a5-b8f1-4c78-9a71-2daa88462959-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8wjlh\" (UID: \"7ca049a5-b8f1-4c78-9a71-2daa88462959\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:38.063920 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:38.063849    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"
Sep 10 11:56:38.372865 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:38.372789    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh"]
Sep 10 11:56:38.377760 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:56:38.377724    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ca049a5_b8f1_4c78_9a71_2daa88462959.slice/crio-6705c361b04c4ea67ee863c168cc478f89e3f9686a232d2d809ce25ba45bab14 WatchSource:0}: Error finding container 6705c361b04c4ea67ee863c168cc478f89e3f9686a232d2d809ce25ba45bab14: Status 404 returned error can't find the container with id 6705c361b04c4ea67ee863c168cc478f89e3f9686a232d2d809ce25ba45bab14
Sep 10 11:56:39.108009 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:39.107968    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh" event={"ID":"7ca049a5-b8f1-4c78-9a71-2daa88462959","Type":"ContainerStarted","Data":"6705c361b04c4ea67ee863c168cc478f89e3f9686a232d2d809ce25ba45bab14"}
Sep 10 11:56:39.694679 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:39.694634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:39.793727 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:39.793694    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:40.114697 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:40.114615    2571 generic.go:356] "Generic (PLEG): container finished" podID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerID="bd5689cb366c03a6c3e07e915ba3255f66a603c9f7e66bb38e7758cb3e92cd43" exitCode=0
Sep 10 11:56:40.114697 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:40.114669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr" event={"ID":"10cf5c11-aa9a-4069-abc3-31aee27819d3","Type":"ContainerDied","Data":"bd5689cb366c03a6c3e07e915ba3255f66a603c9f7e66bb38e7758cb3e92cd43"}
Sep 10 11:56:41.122102 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.122063    2571 generic.go:356] "Generic (PLEG): container finished" podID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerID="2561d152cfd5e00fcf1616fb3e57ec98c269b18a9b298b739f6fabda8c5e53d7" exitCode=0
Sep 10 11:56:41.122579 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.122142    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr" event={"ID":"10cf5c11-aa9a-4069-abc3-31aee27819d3","Type":"ContainerDied","Data":"2561d152cfd5e00fcf1616fb3e57ec98c269b18a9b298b739f6fabda8c5e53d7"}
Sep 10 11:56:41.471418 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.471386    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d9r75"]
Sep 10 11:56:41.473564 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.473542    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.476376 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.476355    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fsbjg\""
Sep 10 11:56:41.484982 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.484952    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d9r75"]
Sep 10 11:56:41.574332 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.574298    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e9b36d0-e803-4d34-8931-e6c4768806f9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d9r75\" (UID: \"5e9b36d0-e803-4d34-8931-e6c4768806f9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.574332 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.574334    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7v5n\" (UniqueName: \"kubernetes.io/projected/5e9b36d0-e803-4d34-8931-e6c4768806f9-kube-api-access-x7v5n\") pod \"cert-manager-webhook-6486ffd997-d9r75\" (UID: \"5e9b36d0-e803-4d34-8931-e6c4768806f9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.675369 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.675334    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e9b36d0-e803-4d34-8931-e6c4768806f9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d9r75\" (UID: \"5e9b36d0-e803-4d34-8931-e6c4768806f9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.675369 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.675372    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7v5n\" (UniqueName: \"kubernetes.io/projected/5e9b36d0-e803-4d34-8931-e6c4768806f9-kube-api-access-x7v5n\") pod \"cert-manager-webhook-6486ffd997-d9r75\" (UID: \"5e9b36d0-e803-4d34-8931-e6c4768806f9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.684088 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.684057    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5e9b36d0-e803-4d34-8931-e6c4768806f9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-d9r75\" (UID: \"5e9b36d0-e803-4d34-8931-e6c4768806f9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.684220 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.684173    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7v5n\" (UniqueName: \"kubernetes.io/projected/5e9b36d0-e803-4d34-8931-e6c4768806f9-kube-api-access-x7v5n\") pod \"cert-manager-webhook-6486ffd997-d9r75\" (UID: \"5e9b36d0-e803-4d34-8931-e6c4768806f9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:41.782900 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:41.782834    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:42.066718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.066693    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-d9r75"]
Sep 10 11:56:42.068392 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:56:42.068365    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e9b36d0_e803_4d34_8931_e6c4768806f9.slice/crio-1d9771a53d2f4c741d37c5b88d495c425cf5fcc36b1579c742365ad27f2ca462 WatchSource:0}: Error finding container 1d9771a53d2f4c741d37c5b88d495c425cf5fcc36b1579c742365ad27f2ca462: Status 404 returned error can't find the container with id 1d9771a53d2f4c741d37c5b88d495c425cf5fcc36b1579c742365ad27f2ca462
Sep 10 11:56:42.128449 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.128413    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh" event={"ID":"7ca049a5-b8f1-4c78-9a71-2daa88462959","Type":"ContainerStarted","Data":"df365a736b7995119702b768c221cf528570d08381848ee592ca852922287d5b"}
Sep 10 11:56:42.129994 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.129968    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75" event={"ID":"5e9b36d0-e803-4d34-8931-e6c4768806f9","Type":"ContainerStarted","Data":"d7130f6b433d0afcc7e9ae1d67e8239281c104a4dc166ae39b29b54b5eed21d4"}
Sep 10 11:56:42.130093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.130008    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75" event={"ID":"5e9b36d0-e803-4d34-8931-e6c4768806f9","Type":"ContainerStarted","Data":"1d9771a53d2f4c741d37c5b88d495c425cf5fcc36b1579c742365ad27f2ca462"}
Sep 10 11:56:42.130093 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.130056    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:42.157338 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.157287    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8wjlh" podStartSLOduration=2.300313152 podStartE2EDuration="5.157270016s" totalImagesPullingTime="2.856956864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:56:42.156523651 +0000 UTC m=+312.759798562" watchObservedRunningTime="2026-09-10 11:56:42.157270016 +0000 UTC m=+312.760544926"
Sep 10 11:56:42.180592 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.180523    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75" podStartSLOduration=1.180499469 podStartE2EDuration="1.180499469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:56:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:56:42.178230271 +0000 UTC m=+312.781505183" watchObservedRunningTime="2026-09-10 11:56:42.180499469 +0000 UTC m=+312.783774381"
Sep 10 11:56:42.248382 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.248358    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:42.380764 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.380679    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwb4c\" (UniqueName: \"kubernetes.io/projected/10cf5c11-aa9a-4069-abc3-31aee27819d3-kube-api-access-pwb4c\") pod \"10cf5c11-aa9a-4069-abc3-31aee27819d3\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") "
Sep 10 11:56:42.380905 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.380725    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-bundle\") pod \"10cf5c11-aa9a-4069-abc3-31aee27819d3\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") "
Sep 10 11:56:42.381872 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.381198    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-util\") pod \"10cf5c11-aa9a-4069-abc3-31aee27819d3\" (UID: \"10cf5c11-aa9a-4069-abc3-31aee27819d3\") "
Sep 10 11:56:42.381872 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.381797    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-bundle" (OuterVolumeSpecName: "bundle") pod "10cf5c11-aa9a-4069-abc3-31aee27819d3" (UID: "10cf5c11-aa9a-4069-abc3-31aee27819d3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:56:42.383227 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.382358    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:56:42.386537 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.383581    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10cf5c11-aa9a-4069-abc3-31aee27819d3-kube-api-access-pwb4c" (OuterVolumeSpecName: "kube-api-access-pwb4c") pod "10cf5c11-aa9a-4069-abc3-31aee27819d3" (UID: "10cf5c11-aa9a-4069-abc3-31aee27819d3"). InnerVolumeSpecName "kube-api-access-pwb4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:56:42.387326 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.387302    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-util" (OuterVolumeSpecName: "util") pod "10cf5c11-aa9a-4069-abc3-31aee27819d3" (UID: "10cf5c11-aa9a-4069-abc3-31aee27819d3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:56:42.482869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.482841    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwb4c\" (UniqueName: \"kubernetes.io/projected/10cf5c11-aa9a-4069-abc3-31aee27819d3-kube-api-access-pwb4c\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:56:42.482869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:42.482865    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10cf5c11-aa9a-4069-abc3-31aee27819d3-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:56:43.136636 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:43.136603    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr"
Sep 10 11:56:43.137045 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:43.136650    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzpwsr" event={"ID":"10cf5c11-aa9a-4069-abc3-31aee27819d3","Type":"ContainerDied","Data":"6f5a2320b88f9a941cc03f369069affb7a9947def3e62c979cee0ce29a4fdb42"}
Sep 10 11:56:43.137045 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:43.136699    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f5a2320b88f9a941cc03f369069affb7a9947def3e62c979cee0ce29a4fdb42"
Sep 10 11:56:44.910890 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:44.910856    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:46.722849 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:46.722799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:48.138903 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:48.138871    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-d9r75"
Sep 10 11:56:48.426592 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:48.426514    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:48.525479 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:48.525450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:54.604427 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:54.604388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:54.705382 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:54.705347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:56.186589 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186556    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"]
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186819    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="util"
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186831    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="util"
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186838    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="extract"
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186844    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="extract"
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186857    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="pull"
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186863    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="pull"
Sep 10 11:56:56.186964 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.186903    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="10cf5c11-aa9a-4069-abc3-31aee27819d3" containerName="extract"
Sep 10 11:56:56.188546 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.188530    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.191024 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.190997    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 11:56:56.191989 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.191966    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 11:56:56.192111 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.191992    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8dsfc\""
Sep 10 11:56:56.198941 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.198923    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"]
Sep 10 11:56:56.280988 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.280959    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.281106 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.280994    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.281106 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.281067    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxc5m\" (UniqueName: \"kubernetes.io/projected/bcf31043-3f20-4678-8c8e-0564607f92be-kube-api-access-xxc5m\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.382173 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.382130    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxc5m\" (UniqueName: \"kubernetes.io/projected/bcf31043-3f20-4678-8c8e-0564607f92be-kube-api-access-xxc5m\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.382320 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.382285    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.382320 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.382311    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.382641 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.382616    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.382700 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.382629    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.390680 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.390649    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxc5m\" (UniqueName: \"kubernetes.io/projected/bcf31043-3f20-4678-8c8e-0564607f92be-kube-api-access-xxc5m\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.421434 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.421410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:56:56.497029 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.496945    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:56:56.771604 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:56.771582    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"]
Sep 10 11:56:56.773963 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:56:56.773936    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf31043_3f20_4678_8c8e_0564607f92be.slice/crio-fc0c41ea6194f680afc1b7ac54ace04956aacd81e4d310eb1f5845ee37e35b6d WatchSource:0}: Error finding container fc0c41ea6194f680afc1b7ac54ace04956aacd81e4d310eb1f5845ee37e35b6d: Status 404 returned error can't find the container with id fc0c41ea6194f680afc1b7ac54ace04956aacd81e4d310eb1f5845ee37e35b6d
Sep 10 11:56:57.191131 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:57.191097    2571 generic.go:356] "Generic (PLEG): container finished" podID="bcf31043-3f20-4678-8c8e-0564607f92be" containerID="1d205d8c233ab8f5ec9ed0b235e3feb59f460918270bdfb39ea8b5bb66a762ba" exitCode=0
Sep 10 11:56:57.191538 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:57.191186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv" event={"ID":"bcf31043-3f20-4678-8c8e-0564607f92be","Type":"ContainerDied","Data":"1d205d8c233ab8f5ec9ed0b235e3feb59f460918270bdfb39ea8b5bb66a762ba"}
Sep 10 11:56:57.191538 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:57.191232    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv" event={"ID":"bcf31043-3f20-4678-8c8e-0564607f92be","Type":"ContainerStarted","Data":"fc0c41ea6194f680afc1b7ac54ace04956aacd81e4d310eb1f5845ee37e35b6d"}
Sep 10 11:56:58.196752 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:58.196720    2571 generic.go:356] "Generic (PLEG): container finished" podID="bcf31043-3f20-4678-8c8e-0564607f92be" containerID="670c1d3d1fbb7627b61f620e2647c7b98961a6bc2a6724ad01eb22d18f32aff4" exitCode=0
Sep 10 11:56:58.197101 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:58.196797    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv" event={"ID":"bcf31043-3f20-4678-8c8e-0564607f92be","Type":"ContainerDied","Data":"670c1d3d1fbb7627b61f620e2647c7b98961a6bc2a6724ad01eb22d18f32aff4"}
Sep 10 11:56:59.203664 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:59.203631    2571 generic.go:356] "Generic (PLEG): container finished" podID="bcf31043-3f20-4678-8c8e-0564607f92be" containerID="6c1119e93eb0d8a6c38c681f6836f2c40a7e2df47c2db10c6f31e18f9eee3c61" exitCode=0
Sep 10 11:56:59.204046 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:56:59.203705    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv" event={"ID":"bcf31043-3f20-4678-8c8e-0564607f92be","Type":"ContainerDied","Data":"6c1119e93eb0d8a6c38c681f6836f2c40a7e2df47c2db10c6f31e18f9eee3c61"}
Sep 10 11:57:00.322621 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.322599    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:57:00.413162 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.413132    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-bundle\") pod \"bcf31043-3f20-4678-8c8e-0564607f92be\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") "
Sep 10 11:57:00.413314 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.413219    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxc5m\" (UniqueName: \"kubernetes.io/projected/bcf31043-3f20-4678-8c8e-0564607f92be-kube-api-access-xxc5m\") pod \"bcf31043-3f20-4678-8c8e-0564607f92be\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") "
Sep 10 11:57:00.413314 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.413249    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-util\") pod \"bcf31043-3f20-4678-8c8e-0564607f92be\" (UID: \"bcf31043-3f20-4678-8c8e-0564607f92be\") "
Sep 10 11:57:00.413897 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.413872    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-bundle" (OuterVolumeSpecName: "bundle") pod "bcf31043-3f20-4678-8c8e-0564607f92be" (UID: "bcf31043-3f20-4678-8c8e-0564607f92be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:57:00.415302 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.415274    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf31043-3f20-4678-8c8e-0564607f92be-kube-api-access-xxc5m" (OuterVolumeSpecName: "kube-api-access-xxc5m") pod "bcf31043-3f20-4678-8c8e-0564607f92be" (UID: "bcf31043-3f20-4678-8c8e-0564607f92be"). InnerVolumeSpecName "kube-api-access-xxc5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:57:00.418657 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.418620    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-util" (OuterVolumeSpecName: "util") pod "bcf31043-3f20-4678-8c8e-0564607f92be" (UID: "bcf31043-3f20-4678-8c8e-0564607f92be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:57:00.513687 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.513609    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:57:00.513687 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.513635    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxc5m\" (UniqueName: \"kubernetes.io/projected/bcf31043-3f20-4678-8c8e-0564607f92be-kube-api-access-xxc5m\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:57:00.513687 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:00.513649    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bcf31043-3f20-4678-8c8e-0564607f92be-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:57:01.213507 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:01.213471    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv" event={"ID":"bcf31043-3f20-4678-8c8e-0564607f92be","Type":"ContainerDied","Data":"fc0c41ea6194f680afc1b7ac54ace04956aacd81e4d310eb1f5845ee37e35b6d"}
Sep 10 11:57:01.213736 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:01.213513    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwdpjv"
Sep 10 11:57:01.213736 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:01.213522    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc0c41ea6194f680afc1b7ac54ace04956aacd81e4d310eb1f5845ee37e35b6d"
Sep 10 11:57:10.923539 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923501    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"]
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923761    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="pull"
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923773    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="pull"
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923790    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="util"
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923796    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="util"
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923808    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="extract"
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923814    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="extract"
Sep 10 11:57:10.923917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.923854    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf31043-3f20-4678-8c8e-0564607f92be" containerName="extract"
Sep 10 11:57:10.971292 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.971261    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"]
Sep 10 11:57:10.971439 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.971387    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:10.974857 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.974836    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 11:57:10.975528 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.975512    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8dsfc\""
Sep 10 11:57:10.975634 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:10.975551    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 11:57:11.085047 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.085013    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbrfv\" (UniqueName: \"kubernetes.io/projected/ce36ea20-3e7f-4141-8905-4b483b2a3495-kube-api-access-gbrfv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.085047 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.085059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.085297 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.085078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.186140 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.186109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbrfv\" (UniqueName: \"kubernetes.io/projected/ce36ea20-3e7f-4141-8905-4b483b2a3495-kube-api-access-gbrfv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.186322 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.186180    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.186322 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.186210    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.186550 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.186529    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.186619 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.186566    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.201612 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.201585    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbrfv\" (UniqueName: \"kubernetes.io/projected/ce36ea20-3e7f-4141-8905-4b483b2a3495-kube-api-access-gbrfv\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.279714 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.279685    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:11.582890 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:11.582866    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"]
Sep 10 11:57:11.585129 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:57:11.585104    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce36ea20_3e7f_4141_8905_4b483b2a3495.slice/crio-968c84041ff8a452ddd143027849196624ae93ca173f344e05de66e4eeef6006 WatchSource:0}: Error finding container 968c84041ff8a452ddd143027849196624ae93ca173f344e05de66e4eeef6006: Status 404 returned error can't find the container with id 968c84041ff8a452ddd143027849196624ae93ca173f344e05de66e4eeef6006
Sep 10 11:57:12.207927 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:12.207901    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:12.260211 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:12.260148    2571 generic.go:356] "Generic (PLEG): container finished" podID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerID="83750d18457b34723312fc1793c29a2740a1d41db3f099c40b30e7f2eca2dfae" exitCode=0
Sep 10 11:57:12.260425 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:12.260244    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf" event={"ID":"ce36ea20-3e7f-4141-8905-4b483b2a3495","Type":"ContainerDied","Data":"83750d18457b34723312fc1793c29a2740a1d41db3f099c40b30e7f2eca2dfae"}
Sep 10 11:57:12.260425 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:12.260281    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf" event={"ID":"ce36ea20-3e7f-4141-8905-4b483b2a3495","Type":"ContainerStarted","Data":"968c84041ff8a452ddd143027849196624ae93ca173f344e05de66e4eeef6006"}
Sep 10 11:57:13.115545 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.115509    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"]
Sep 10 11:57:13.118324 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.118308    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.122504 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.122474    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 11:57:13.122626 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.122513    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 11:57:13.122626 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.122574    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-64jsz\""
Sep 10 11:57:13.122717 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.122637    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 11:57:13.122906 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.122890    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 11:57:13.147572 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.147511    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"]
Sep 10 11:57:13.271204 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.271166    2571 generic.go:356] "Generic (PLEG): container finished" podID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerID="259b6cbc8a789f7fd5a3b27d9cd10c039181d712d03132be68440011e90d3f78" exitCode=0
Sep 10 11:57:13.271656 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.271252    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf" event={"ID":"ce36ea20-3e7f-4141-8905-4b483b2a3495","Type":"ContainerDied","Data":"259b6cbc8a789f7fd5a3b27d9cd10c039181d712d03132be68440011e90d3f78"}
Sep 10 11:57:13.301851 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.301824    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnhfn\" (UniqueName: \"kubernetes.io/projected/4df95b75-8775-4c56-a5f7-7db80b0c956f-kube-api-access-fnhfn\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.301971 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.301876    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df95b75-8775-4c56-a5f7-7db80b0c956f-webhook-cert\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.301971 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.301921    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4df95b75-8775-4c56-a5f7-7db80b0c956f-manifests\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.301971 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.301950    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4df95b75-8775-4c56-a5f7-7db80b0c956f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.301971 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.301973    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4df95b75-8775-4c56-a5f7-7db80b0c956f-tmp\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.402585 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.402502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df95b75-8775-4c56-a5f7-7db80b0c956f-webhook-cert\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.402585 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.402570    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4df95b75-8775-4c56-a5f7-7db80b0c956f-manifests\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.402808 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.402611    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4df95b75-8775-4c56-a5f7-7db80b0c956f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.402808 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.402730    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4df95b75-8775-4c56-a5f7-7db80b0c956f-tmp\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.402808 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.402784    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnhfn\" (UniqueName: \"kubernetes.io/projected/4df95b75-8775-4c56-a5f7-7db80b0c956f-kube-api-access-fnhfn\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.403296 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.403265    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4df95b75-8775-4c56-a5f7-7db80b0c956f-manifests\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.405434 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.405406    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4df95b75-8775-4c56-a5f7-7db80b0c956f-webhook-cert\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.405529 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.405448    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4df95b75-8775-4c56-a5f7-7db80b0c956f-tmp\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.405849 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.405830    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4df95b75-8775-4c56-a5f7-7db80b0c956f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.410909 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.410883    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnhfn\" (UniqueName: \"kubernetes.io/projected/4df95b75-8775-4c56-a5f7-7db80b0c956f-kube-api-access-fnhfn\") pod \"opendatahub-operator-controller-manager-d86dfbd7c-xjg9v\" (UID: \"4df95b75-8775-4c56-a5f7-7db80b0c956f\") " pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.427792 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.427757    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:13.707776 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:13.707746    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"]
Sep 10 11:57:13.710391 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:57:13.710361    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df95b75_8775_4c56_a5f7_7db80b0c956f.slice/crio-343d68db2a932c381fcbc05210aadbfd9725c17c4246907c6ec05780394235b4 WatchSource:0}: Error finding container 343d68db2a932c381fcbc05210aadbfd9725c17c4246907c6ec05780394235b4: Status 404 returned error can't find the container with id 343d68db2a932c381fcbc05210aadbfd9725c17c4246907c6ec05780394235b4
Sep 10 11:57:14.276050 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:14.276010    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v" event={"ID":"4df95b75-8775-4c56-a5f7-7db80b0c956f","Type":"ContainerStarted","Data":"343d68db2a932c381fcbc05210aadbfd9725c17c4246907c6ec05780394235b4"}
Sep 10 11:57:14.278126 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:14.278101    2571 generic.go:356] "Generic (PLEG): container finished" podID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerID="e884ef8aa7c29474709119e95017fd06d69e0e7325a2afab10b084a0d8a5861c" exitCode=0
Sep 10 11:57:14.278253 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:14.278143    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf" event={"ID":"ce36ea20-3e7f-4141-8905-4b483b2a3495","Type":"ContainerDied","Data":"e884ef8aa7c29474709119e95017fd06d69e0e7325a2afab10b084a0d8a5861c"}
Sep 10 11:57:15.022503 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:15.022459    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:16.722018 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.721995    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:16.831266 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.831241    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-bundle\") pod \"ce36ea20-3e7f-4141-8905-4b483b2a3495\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") "
Sep 10 11:57:16.831392 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.831284    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbrfv\" (UniqueName: \"kubernetes.io/projected/ce36ea20-3e7f-4141-8905-4b483b2a3495-kube-api-access-gbrfv\") pod \"ce36ea20-3e7f-4141-8905-4b483b2a3495\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") "
Sep 10 11:57:16.831392 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.831306    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-util\") pod \"ce36ea20-3e7f-4141-8905-4b483b2a3495\" (UID: \"ce36ea20-3e7f-4141-8905-4b483b2a3495\") "
Sep 10 11:57:16.831956 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.831928    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-bundle" (OuterVolumeSpecName: "bundle") pod "ce36ea20-3e7f-4141-8905-4b483b2a3495" (UID: "ce36ea20-3e7f-4141-8905-4b483b2a3495"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:57:16.833112 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.833091    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce36ea20-3e7f-4141-8905-4b483b2a3495-kube-api-access-gbrfv" (OuterVolumeSpecName: "kube-api-access-gbrfv") pod "ce36ea20-3e7f-4141-8905-4b483b2a3495" (UID: "ce36ea20-3e7f-4141-8905-4b483b2a3495"). InnerVolumeSpecName "kube-api-access-gbrfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:57:16.839295 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.839270    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-util" (OuterVolumeSpecName: "util") pod "ce36ea20-3e7f-4141-8905-4b483b2a3495" (UID: "ce36ea20-3e7f-4141-8905-4b483b2a3495"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:57:16.932246 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.932208    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:57:16.932246 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.932240    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbrfv\" (UniqueName: \"kubernetes.io/projected/ce36ea20-3e7f-4141-8905-4b483b2a3495-kube-api-access-gbrfv\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:57:16.932246 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:16.932249    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce36ea20-3e7f-4141-8905-4b483b2a3495-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:57:17.292858 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:17.292813    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v" event={"ID":"4df95b75-8775-4c56-a5f7-7db80b0c956f","Type":"ContainerStarted","Data":"26cdd7b44baecd7b8a89e8bdd864d4d8c1917912bfe47646a482b7e3f7bf350e"}
Sep 10 11:57:17.294692 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:17.294669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf" event={"ID":"ce36ea20-3e7f-4141-8905-4b483b2a3495","Type":"ContainerDied","Data":"968c84041ff8a452ddd143027849196624ae93ca173f344e05de66e4eeef6006"}
Sep 10 11:57:17.294833 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:17.294703    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="968c84041ff8a452ddd143027849196624ae93ca173f344e05de66e4eeef6006"
Sep 10 11:57:17.294833 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:17.294722    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08qcvf"
Sep 10 11:57:18.300236 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:18.300203    2571 generic.go:356] "Generic (PLEG): container finished" podID="4df95b75-8775-4c56-a5f7-7db80b0c956f" containerID="26cdd7b44baecd7b8a89e8bdd864d4d8c1917912bfe47646a482b7e3f7bf350e" exitCode=0
Sep 10 11:57:18.300701 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:18.300274    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v" event={"ID":"4df95b75-8775-4c56-a5f7-7db80b0c956f","Type":"ContainerDied","Data":"26cdd7b44baecd7b8a89e8bdd864d4d8c1917912bfe47646a482b7e3f7bf350e"}
Sep 10 11:57:19.305890 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:19.305851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v" event={"ID":"4df95b75-8775-4c56-a5f7-7db80b0c956f","Type":"ContainerStarted","Data":"c09815a3c0fcfe7744008f37da7456cc0ae7a96a638f2af4c4045cf9b68e7971"}
Sep 10 11:57:19.306304 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:19.305976    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:19.331820 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:19.331771    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v" podStartSLOduration=1.6829083009999999 podStartE2EDuration="6.331755186s" totalImagesPullingTime="3.583593391s" totalInitContainerRuntime="1.065253494s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:57:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:57:19.330859562 +0000 UTC m=+349.934134488" watchObservedRunningTime="2026-09-10 11:57:19.331755186 +0000 UTC m=+349.935030097"
Sep 10 11:57:25.038249 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:25.038144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:25.132701 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:25.132666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:30.312235 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:30.312201    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d86dfbd7c-xjg9v"
Sep 10 11:57:32.139573 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:32.139539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:32.297025 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:32.296989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:41.521118 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521083    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"]
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521348    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="pull"
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521359    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="pull"
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521372    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="util"
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521377    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="util"
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521384    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="extract"
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521390    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="extract"
Sep 10 11:57:41.521541 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.521436    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce36ea20-3e7f-4141-8905-4b483b2a3495" containerName="extract"
Sep 10 11:57:41.526013 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.525995    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.528568 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.528547    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 11:57:41.530027 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.530007    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wcbp2\""
Sep 10 11:57:41.530027 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.530020    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:57:41.530220 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.530009    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 11:57:41.530220 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.530055    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 11:57:41.530220 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.530075    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 11:57:41.536293 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.536271    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"]
Sep 10 11:57:41.610681 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.610649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-manager-config\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.610842 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.610695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-metrics-cert\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.610842 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.610729    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-cert\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.610842 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.610776    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjm2\" (UniqueName: \"kubernetes.io/projected/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-kube-api-access-2vjm2\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.711670 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.711633    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-manager-config\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.711869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.711686    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-metrics-cert\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.711869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.711708    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-cert\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.711869 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.711740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjm2\" (UniqueName: \"kubernetes.io/projected/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-kube-api-access-2vjm2\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.712476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.712455    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-manager-config\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.714330 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.714298    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-metrics-cert\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.714421 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.714304    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-cert\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.722205 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.722184    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjm2\" (UniqueName: \"kubernetes.io/projected/e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77-kube-api-access-2vjm2\") pod \"lws-controller-manager-5f7f7d474c-6xt7d\" (UID: \"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77\") " pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:41.834998 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:41.834910    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:42.109542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:42.109517    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"]
Sep 10 11:57:42.111944 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:57:42.111916    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fa3e7f_a6a7_4c3d_845a_c9cc0861cf77.slice/crio-fb390f6746e3f564e66a0484978e19ed9a7a8fbb6782247c05fc3437d426725a WatchSource:0}: Error finding container fb390f6746e3f564e66a0484978e19ed9a7a8fbb6782247c05fc3437d426725a: Status 404 returned error can't find the container with id fb390f6746e3f564e66a0484978e19ed9a7a8fbb6782247c05fc3437d426725a
Sep 10 11:57:42.408868 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:42.408785    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d" event={"ID":"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77","Type":"ContainerStarted","Data":"fb390f6746e3f564e66a0484978e19ed9a7a8fbb6782247c05fc3437d426725a"}
Sep 10 11:57:43.016848 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:43.016805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:43.262974 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:43.262928    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:43.360906 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:43.360821    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:43.688189 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:43.688124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:43.805007 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:43.804970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:44.419667 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:44.419635    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d" event={"ID":"e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77","Type":"ContainerStarted","Data":"392a284856a4acb549472aea1c7eef8850054b23b15eb9f8a4eaaed740c4f354"}
Sep 10 11:57:44.420115 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:44.419703    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:44.444565 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:44.444517    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d" podStartSLOduration=1.970375236 podStartE2EDuration="3.444501583s" totalImagesPullingTime="1.474126347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:57:44.443193762 +0000 UTC m=+375.046468673" watchObservedRunningTime="2026-09-10 11:57:44.444501583 +0000 UTC m=+375.047776494"
Sep 10 11:57:44.753007 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:44.752973    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:44.830600 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:44.830571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:46.098340 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:46.098306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:51.728479 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:51.728442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:51.862248 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:51.862214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:52.026166 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.026079    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"]
Sep 10 11:57:52.032090 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.032068    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.034856 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.034834    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 11:57:52.034990 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.034924    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 11:57:52.034990 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.034924    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 11:57:52.035082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.034927    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fzjfz\""
Sep 10 11:57:52.059876 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.059851    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"]
Sep 10 11:57:52.090573 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090536    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090573 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090751 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090595    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090751 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090682    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090751 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090707    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090866 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090765    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhc5f\" (UniqueName: \"kubernetes.io/projected/05ac0d9e-4686-4505-8faf-0ffe1cd91361-kube-api-access-mhc5f\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090866 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090866 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.090866 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.090863    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.095469 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.095446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:52.191730 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhc5f\" (UniqueName: \"kubernetes.io/projected/05ac0d9e-4686-4505-8faf-0ffe1cd91361-kube-api-access-mhc5f\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.191730 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191735    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.191974 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.191974 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.191974 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191825    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.191974 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191842    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192212 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.191977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192212 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192046    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192212 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192377 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192337    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192377 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192349    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192419    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192694 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192677    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.192853 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.192837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.194207 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.194184    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.194354 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.194338    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.206907 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.206882    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05ac0d9e-4686-4505-8faf-0ffe1cd91361-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.210761 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.210739    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhc5f\" (UniqueName: \"kubernetes.io/projected/05ac0d9e-4686-4505-8faf-0ffe1cd91361-kube-api-access-mhc5f\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd\" (UID: \"05ac0d9e-4686-4505-8faf-0ffe1cd91361\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.341807 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.341726    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:52.385388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.385353    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"]
Sep 10 11:57:52.388680 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.388651    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.392844 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.391854    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 11:57:52.392844 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.391943    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 11:57:52.392844 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.392037    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 11:57:52.392844 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.392110    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r86vv\""
Sep 10 11:57:52.409466 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.409127    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"]
Sep 10 11:57:52.410137 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.410064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:52.493950 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.493922    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.494125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.493968    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.494125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.494031    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.494125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.494068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw6wc\" (UniqueName: \"kubernetes.io/projected/622567f6-4603-45ce-92c2-a2fc38a99e88-kube-api-access-hw6wc\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.494125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.494096    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/622567f6-4603-45ce-92c2-a2fc38a99e88-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.494125 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.494116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.494335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.494200    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.519122 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.519097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:52.595446 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595384    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.595446 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw6wc\" (UniqueName: \"kubernetes.io/projected/622567f6-4603-45ce-92c2-a2fc38a99e88-kube-api-access-hw6wc\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.595446 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595435    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/622567f6-4603-45ce-92c2-a2fc38a99e88-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.595673 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595580    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.595730 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595679    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.595788 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.595837 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.595809    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.596548 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.596511    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.597650 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.597629    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/622567f6-4603-45ce-92c2-a2fc38a99e88-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.598236 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.598211    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.598422 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.598403    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.598487 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.598396    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.604436 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.604418    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/622567f6-4603-45ce-92c2-a2fc38a99e88-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.607867 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.607849    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw6wc\" (UniqueName: \"kubernetes.io/projected/622567f6-4603-45ce-92c2-a2fc38a99e88-kube-api-access-hw6wc\") pod \"istiod-openshift-gateway-5677ccb49b-qnbd7\" (UID: \"622567f6-4603-45ce-92c2-a2fc38a99e88\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.621874 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:57:52.621855    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ac0d9e_4686_4505_8faf_0ffe1cd91361.slice/crio-02123b034849d92f0a1ec888a27e5a7d208682c30532e091f29f7fdc16b5c7ae WatchSource:0}: Error finding container 02123b034849d92f0a1ec888a27e5a7d208682c30532e091f29f7fdc16b5c7ae: Status 404 returned error can't find the container with id 02123b034849d92f0a1ec888a27e5a7d208682c30532e091f29f7fdc16b5c7ae
Sep 10 11:57:52.621874 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.621858    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"]
Sep 10 11:57:52.700919 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.700895    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:52.992479 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:52.992444    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"]
Sep 10 11:57:52.995280 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:57:52.995256    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622567f6_4603_45ce_92c2_a2fc38a99e88.slice/crio-ce0c8b3c69f6f2bc5d420734094dd8aa9eccf02232ceff8e458c10736440f623 WatchSource:0}: Error finding container ce0c8b3c69f6f2bc5d420734094dd8aa9eccf02232ceff8e458c10736440f623: Status 404 returned error can't find the container with id ce0c8b3c69f6f2bc5d420734094dd8aa9eccf02232ceff8e458c10736440f623
Sep 10 11:57:53.456062 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:53.456027    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd" event={"ID":"05ac0d9e-4686-4505-8faf-0ffe1cd91361","Type":"ContainerStarted","Data":"02123b034849d92f0a1ec888a27e5a7d208682c30532e091f29f7fdc16b5c7ae"}
Sep 10 11:57:53.457528 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:53.457496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7" event={"ID":"622567f6-4603-45ce-92c2-a2fc38a99e88","Type":"ContainerStarted","Data":"ce0c8b3c69f6f2bc5d420734094dd8aa9eccf02232ceff8e458c10736440f623"}
Sep 10 11:57:55.409793 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:55.409740    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:55.425430 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:55.425403    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5f7f7d474c-6xt7d"
Sep 10 11:57:55.560712 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:55.560679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:55.756542 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:55.756506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:56.492498 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.492450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:56.577791 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.577744    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 11:57:56.577922 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.577830    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 11:57:56.577922 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.577858    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 11:57:56.579849 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.579826    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 11:57:56.579909 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.579876    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 11:57:56.594547 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:56.594522    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:57:57.482220 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:57.482186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd" event={"ID":"05ac0d9e-4686-4505-8faf-0ffe1cd91361","Type":"ContainerStarted","Data":"8d51adf93ceba69bb23645b22c088046849c7f1865eb6a8c4c9c0a69a51fd9a7"}
Sep 10 11:57:57.483686 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:57.483658    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7" event={"ID":"622567f6-4603-45ce-92c2-a2fc38a99e88","Type":"ContainerStarted","Data":"44a41c8f080eab49adfa059f3bed066f5cc02ed4ac392876ed9ff2bfc329ffdb"}
Sep 10 11:57:57.483808 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:57.483711    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:57.485446 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:57.485425    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7"
Sep 10 11:57:57.530118 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:57.530073    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd" podStartSLOduration=2.576309495 podStartE2EDuration="6.530057793s" totalImagesPullingTime="3.953748298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:57:57.528482177 +0000 UTC m=+388.131757090" watchObservedRunningTime="2026-09-10 11:57:57.530057793 +0000 UTC m=+388.133332704"
Sep 10 11:57:57.573168 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:57.573116    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-qnbd7" podStartSLOduration=1.990740201 podStartE2EDuration="5.573102545s" totalImagesPullingTime="3.582362344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:57:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:57:57.571810212 +0000 UTC m=+388.175085123" watchObservedRunningTime="2026-09-10 11:57:57.573102545 +0000 UTC m=+388.176377456"
Sep 10 11:57:58.341853 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:58.341816    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:58.346374 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:58.346350    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:58.488395 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:58.488361    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:57:58.489246 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:57:58.489228    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd"
Sep 10 11:58:21.787048 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:21.787015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:21.858142 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:21.858109    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:32.548826 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:32.548794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:32.649759 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:32.649729    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:32.748787 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:32.748759    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:44.434061 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:44.434029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:44.520731 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:44.520698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:58:49.291708 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.291668    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"]
Sep 10 11:58:49.301893 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.301872    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.304550 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.304526    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 11:58:49.304804 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.304781    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"]
Sep 10 11:58:49.305488 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.305468    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 11:58:49.305625 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.305607    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8dsfc\""
Sep 10 11:58:49.352441 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.352414    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.352553 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.352454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68ptc\" (UniqueName: \"kubernetes.io/projected/680e81a3-8ef0-4029-a6a0-625c24882954-kube-api-access-68ptc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.352553 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.352513    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.399365 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.399341    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"]
Sep 10 11:58:49.402545 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.402530    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.412431 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.412409    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"]
Sep 10 11:58:49.453417 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453395    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.453531 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.453531 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453461    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.453531 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453505    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.453662 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453544    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdz9l\" (UniqueName: \"kubernetes.io/projected/984b48f8-14d0-4162-966a-301606e89af7-kube-api-access-kdz9l\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.453662 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68ptc\" (UniqueName: \"kubernetes.io/projected/680e81a3-8ef0-4029-a6a0-625c24882954-kube-api-access-68ptc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.453845 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453825    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.453895 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.453837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.462739 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.462713    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68ptc\" (UniqueName: \"kubernetes.io/projected/680e81a3-8ef0-4029-a6a0-625c24882954-kube-api-access-68ptc\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.499613 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.499588    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"]
Sep 10 11:58:49.503077 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.503064    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.512737 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.512710    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"]
Sep 10 11:58:49.554330 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554268    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.554330 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.554487 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554374    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdz9l\" (UniqueName: \"kubernetes.io/projected/984b48f8-14d0-4162-966a-301606e89af7-kube-api-access-kdz9l\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.554487 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554425    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r92p\" (UniqueName: \"kubernetes.io/projected/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-kube-api-access-8r92p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.554487 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554478    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.554635 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554507    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.554687 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554636    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.554768 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.554743    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.562728 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.562700    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdz9l\" (UniqueName: \"kubernetes.io/projected/984b48f8-14d0-4162-966a-301606e89af7-kube-api-access-kdz9l\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.593936 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.593912    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"]
Sep 10 11:58:49.597217 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.597201    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.607022 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.607000    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"]
Sep 10 11:58:49.611660 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.611639    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:49.655047 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655011    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9j6v\" (UniqueName: \"kubernetes.io/projected/890b894a-830c-49a3-8c26-656d9a5a849f-kube-api-access-t9j6v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.655197 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r92p\" (UniqueName: \"kubernetes.io/projected/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-kube-api-access-8r92p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.655270 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.655270 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655227    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.655382 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655269    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.655382 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.655843 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655823    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.655947 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.655867    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.664946 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.664885    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r92p\" (UniqueName: \"kubernetes.io/projected/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-kube-api-access-8r92p\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.710959 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.710932    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:49.756919 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.756805    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.757858 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.757643    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.757858 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.757780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.758055 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.757870    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9j6v\" (UniqueName: \"kubernetes.io/projected/890b894a-830c-49a3-8c26-656d9a5a849f-kube-api-access-t9j6v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.758387 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.758363    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.768076 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.768049    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9j6v\" (UniqueName: \"kubernetes.io/projected/890b894a-830c-49a3-8c26-656d9a5a849f-kube-api-access-t9j6v\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.812935 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.812864    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:49.882484 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.882323    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"]
Sep 10 11:58:49.885067 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:58:49.885042    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680e81a3_8ef0_4029_a6a0_625c24882954.slice/crio-c5ef601d0355a20d2bdad82d18620b6e8f2d27f4b7597714ac8a8d829e1d910a WatchSource:0}: Error finding container c5ef601d0355a20d2bdad82d18620b6e8f2d27f4b7597714ac8a8d829e1d910a: Status 404 returned error can't find the container with id c5ef601d0355a20d2bdad82d18620b6e8f2d27f4b7597714ac8a8d829e1d910a
Sep 10 11:58:49.906592 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.906569    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:49.987399 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:49.987368    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"]
Sep 10 11:58:49.988484 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:58:49.988459    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984b48f8_14d0_4162_966a_301606e89af7.slice/crio-1f257671f6e406354af8f764431f47a289169cad3765416a76b3a3b799d5536d WatchSource:0}: Error finding container 1f257671f6e406354af8f764431f47a289169cad3765416a76b3a3b799d5536d: Status 404 returned error can't find the container with id 1f257671f6e406354af8f764431f47a289169cad3765416a76b3a3b799d5536d
Sep 10 11:58:50.082949 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.082924    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"]
Sep 10 11:58:50.085670 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:58:50.085647    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f14afbc_663d_4e04_8ba8_3e8e2fac4ddc.slice/crio-bb024aeeeb2db4b8f33582e53c3f44e05c136ea6be1b17e65493927dfd5d1bc7 WatchSource:0}: Error finding container bb024aeeeb2db4b8f33582e53c3f44e05c136ea6be1b17e65493927dfd5d1bc7: Status 404 returned error can't find the container with id bb024aeeeb2db4b8f33582e53c3f44e05c136ea6be1b17e65493927dfd5d1bc7
Sep 10 11:58:50.180022 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.179989    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"]
Sep 10 11:58:50.228942 ip-10-0-129-41 kubenswrapper[2571]: W0910 11:58:50.228913    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod890b894a_830c_49a3_8c26_656d9a5a849f.slice/crio-5a5753a4f21e3b1bfc7f3babaae6d9e09e734db61d392794df08da0a88866476 WatchSource:0}: Error finding container 5a5753a4f21e3b1bfc7f3babaae6d9e09e734db61d392794df08da0a88866476: Status 404 returned error can't find the container with id 5a5753a4f21e3b1bfc7f3babaae6d9e09e734db61d392794df08da0a88866476
Sep 10 11:58:50.713700 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.713672    2571 generic.go:356] "Generic (PLEG): container finished" podID="890b894a-830c-49a3-8c26-656d9a5a849f" containerID="de2ca7465047b10bddafb7093d23a2e1965f45780cece22ef997d5fffd2923c5" exitCode=0
Sep 10 11:58:50.714127 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.713755    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf" event={"ID":"890b894a-830c-49a3-8c26-656d9a5a849f","Type":"ContainerDied","Data":"de2ca7465047b10bddafb7093d23a2e1965f45780cece22ef997d5fffd2923c5"}
Sep 10 11:58:50.714127 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.713831    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf" event={"ID":"890b894a-830c-49a3-8c26-656d9a5a849f","Type":"ContainerStarted","Data":"5a5753a4f21e3b1bfc7f3babaae6d9e09e734db61d392794df08da0a88866476"}
Sep 10 11:58:50.715335 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.715311    2571 generic.go:356] "Generic (PLEG): container finished" podID="984b48f8-14d0-4162-966a-301606e89af7" containerID="e6310a96082292abe69104436085c229fff146a667902699bcca366274ab17b8" exitCode=0
Sep 10 11:58:50.715458 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.715406    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s" event={"ID":"984b48f8-14d0-4162-966a-301606e89af7","Type":"ContainerDied","Data":"e6310a96082292abe69104436085c229fff146a667902699bcca366274ab17b8"}
Sep 10 11:58:50.715524 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.715456    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s" event={"ID":"984b48f8-14d0-4162-966a-301606e89af7","Type":"ContainerStarted","Data":"1f257671f6e406354af8f764431f47a289169cad3765416a76b3a3b799d5536d"}
Sep 10 11:58:50.717181 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.717137    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerID="40513c53a3c6d82cd01a204a8978285c5c51234afbc81375c45aa0a985ad7ebd" exitCode=0
Sep 10 11:58:50.717290 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.717193    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v" event={"ID":"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc","Type":"ContainerDied","Data":"40513c53a3c6d82cd01a204a8978285c5c51234afbc81375c45aa0a985ad7ebd"}
Sep 10 11:58:50.717290 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.717242    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v" event={"ID":"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc","Type":"ContainerStarted","Data":"bb024aeeeb2db4b8f33582e53c3f44e05c136ea6be1b17e65493927dfd5d1bc7"}
Sep 10 11:58:50.718841 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.718823    2571 generic.go:356] "Generic (PLEG): container finished" podID="680e81a3-8ef0-4029-a6a0-625c24882954" containerID="f8e72910f535403847c5581129156b2d30582c2075ef9f6972d493e8b00effac" exitCode=0
Sep 10 11:58:50.718906 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.718851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d" event={"ID":"680e81a3-8ef0-4029-a6a0-625c24882954","Type":"ContainerDied","Data":"f8e72910f535403847c5581129156b2d30582c2075ef9f6972d493e8b00effac"}
Sep 10 11:58:50.718906 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:50.718895    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d" event={"ID":"680e81a3-8ef0-4029-a6a0-625c24882954","Type":"ContainerStarted","Data":"c5ef601d0355a20d2bdad82d18620b6e8f2d27f4b7597714ac8a8d829e1d910a"}
Sep 10 11:58:51.726276 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:51.726249    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerID="ffb2bd6ff465f0205cfc816e75a10b4eecfc027f6b650fe7403fa83a291ee5be" exitCode=0
Sep 10 11:58:51.726568 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:51.726300    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v" event={"ID":"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc","Type":"ContainerDied","Data":"ffb2bd6ff465f0205cfc816e75a10b4eecfc027f6b650fe7403fa83a291ee5be"}
Sep 10 11:58:52.732788 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.732754    2571 generic.go:356] "Generic (PLEG): container finished" podID="984b48f8-14d0-4162-966a-301606e89af7" containerID="65fbfcbeae10ede24307e3c5ba641e4aa6c81e973d27e49255c800a878903296" exitCode=0
Sep 10 11:58:52.733245 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.732841    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s" event={"ID":"984b48f8-14d0-4162-966a-301606e89af7","Type":"ContainerDied","Data":"65fbfcbeae10ede24307e3c5ba641e4aa6c81e973d27e49255c800a878903296"}
Sep 10 11:58:52.735100 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.735078    2571 generic.go:356] "Generic (PLEG): container finished" podID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerID="e642279b22c9373e90df75aefc57273c88b73988a0a1b164ca2b27e6560be55e" exitCode=0
Sep 10 11:58:52.735207 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.735174    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v" event={"ID":"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc","Type":"ContainerDied","Data":"e642279b22c9373e90df75aefc57273c88b73988a0a1b164ca2b27e6560be55e"}
Sep 10 11:58:52.743072 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.743047    2571 generic.go:356] "Generic (PLEG): container finished" podID="680e81a3-8ef0-4029-a6a0-625c24882954" containerID="59f0cd097ac5593a25fc83d27e4dde060b43ecbefa5eff566c4accefb72361ed" exitCode=0
Sep 10 11:58:52.743187 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.743125    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d" event={"ID":"680e81a3-8ef0-4029-a6a0-625c24882954","Type":"ContainerDied","Data":"59f0cd097ac5593a25fc83d27e4dde060b43ecbefa5eff566c4accefb72361ed"}
Sep 10 11:58:52.745105 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.745078    2571 generic.go:356] "Generic (PLEG): container finished" podID="890b894a-830c-49a3-8c26-656d9a5a849f" containerID="a665c99505fa23bbf43161ec38fe3528e866b652cc3a73ca8cd34cd65593b374" exitCode=0
Sep 10 11:58:52.745225 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:52.745126    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf" event={"ID":"890b894a-830c-49a3-8c26-656d9a5a849f","Type":"ContainerDied","Data":"a665c99505fa23bbf43161ec38fe3528e866b652cc3a73ca8cd34cd65593b374"}
Sep 10 11:58:53.752023 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.751982    2571 generic.go:356] "Generic (PLEG): container finished" podID="984b48f8-14d0-4162-966a-301606e89af7" containerID="3d3c02f2a99985d44d7c41d4d73883667960398ed5901b7735a3c0772c582ce1" exitCode=0
Sep 10 11:58:53.752465 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.752065    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s" event={"ID":"984b48f8-14d0-4162-966a-301606e89af7","Type":"ContainerDied","Data":"3d3c02f2a99985d44d7c41d4d73883667960398ed5901b7735a3c0772c582ce1"}
Sep 10 11:58:53.754278 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.754258    2571 generic.go:356] "Generic (PLEG): container finished" podID="680e81a3-8ef0-4029-a6a0-625c24882954" containerID="5c522a6f66b470be625212b6f3af857bbefa830def61839c1eaccc5b07a6f6a6" exitCode=0
Sep 10 11:58:53.754377 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.754341    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d" event={"ID":"680e81a3-8ef0-4029-a6a0-625c24882954","Type":"ContainerDied","Data":"5c522a6f66b470be625212b6f3af857bbefa830def61839c1eaccc5b07a6f6a6"}
Sep 10 11:58:53.756441 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.756413    2571 generic.go:356] "Generic (PLEG): container finished" podID="890b894a-830c-49a3-8c26-656d9a5a849f" containerID="0c6caa69998e161d78d9bd25f9278831ea9db2fc78057a214af0e2dfb5d48301" exitCode=0
Sep 10 11:58:53.756571 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.756487    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf" event={"ID":"890b894a-830c-49a3-8c26-656d9a5a849f","Type":"ContainerDied","Data":"0c6caa69998e161d78d9bd25f9278831ea9db2fc78057a214af0e2dfb5d48301"}
Sep 10 11:58:53.878764 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.878742    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:53.990224 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.990188    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-util\") pod \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") "
Sep 10 11:58:53.990388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.990257    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-bundle\") pod \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") "
Sep 10 11:58:53.990388 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.990292    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r92p\" (UniqueName: \"kubernetes.io/projected/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-kube-api-access-8r92p\") pod \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\" (UID: \"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc\") "
Sep 10 11:58:53.990765 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.990733    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-bundle" (OuterVolumeSpecName: "bundle") pod "2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" (UID: "2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:53.992470 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.992447    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-kube-api-access-8r92p" (OuterVolumeSpecName: "kube-api-access-8r92p") pod "2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" (UID: "2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc"). InnerVolumeSpecName "kube-api-access-8r92p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:58:53.995706 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:53.995686    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-util" (OuterVolumeSpecName: "util") pod "2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" (UID: "2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:54.091466 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.091429    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:54.091466 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.091463    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r92p\" (UniqueName: \"kubernetes.io/projected/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-kube-api-access-8r92p\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:54.091466 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.091473    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:54.763234 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.763197    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v"
Sep 10 11:58:54.763234 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.763220    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c2778v" event={"ID":"2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc","Type":"ContainerDied","Data":"bb024aeeeb2db4b8f33582e53c3f44e05c136ea6be1b17e65493927dfd5d1bc7"}
Sep 10 11:58:54.763670 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.763273    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb024aeeeb2db4b8f33582e53c3f44e05c136ea6be1b17e65493927dfd5d1bc7"
Sep 10 11:58:54.904718 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.904685    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:54.948226 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.948205    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:54.950921 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.950901    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:54.999648 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999622    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-bundle\") pod \"890b894a-830c-49a3-8c26-656d9a5a849f\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") "
Sep 10 11:58:54.999811 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999667    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68ptc\" (UniqueName: \"kubernetes.io/projected/680e81a3-8ef0-4029-a6a0-625c24882954-kube-api-access-68ptc\") pod \"680e81a3-8ef0-4029-a6a0-625c24882954\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") "
Sep 10 11:58:54.999811 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999707    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-bundle\") pod \"680e81a3-8ef0-4029-a6a0-625c24882954\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") "
Sep 10 11:58:54.999811 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999737    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-bundle\") pod \"984b48f8-14d0-4162-966a-301606e89af7\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") "
Sep 10 11:58:54.999811 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999766    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9j6v\" (UniqueName: \"kubernetes.io/projected/890b894a-830c-49a3-8c26-656d9a5a849f-kube-api-access-t9j6v\") pod \"890b894a-830c-49a3-8c26-656d9a5a849f\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") "
Sep 10 11:58:54.999811 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999804    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-util\") pod \"680e81a3-8ef0-4029-a6a0-625c24882954\" (UID: \"680e81a3-8ef0-4029-a6a0-625c24882954\") "
Sep 10 11:58:55.000067 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999841    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-util\") pod \"890b894a-830c-49a3-8c26-656d9a5a849f\" (UID: \"890b894a-830c-49a3-8c26-656d9a5a849f\") "
Sep 10 11:58:55.000067 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999872    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdz9l\" (UniqueName: \"kubernetes.io/projected/984b48f8-14d0-4162-966a-301606e89af7-kube-api-access-kdz9l\") pod \"984b48f8-14d0-4162-966a-301606e89af7\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") "
Sep 10 11:58:55.000067 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:54.999940    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-util\") pod \"984b48f8-14d0-4162-966a-301606e89af7\" (UID: \"984b48f8-14d0-4162-966a-301606e89af7\") "
Sep 10 11:58:55.000240 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.000189    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-bundle" (OuterVolumeSpecName: "bundle") pod "984b48f8-14d0-4162-966a-301606e89af7" (UID: "984b48f8-14d0-4162-966a-301606e89af7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:55.000473 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.000448    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-bundle" (OuterVolumeSpecName: "bundle") pod "680e81a3-8ef0-4029-a6a0-625c24882954" (UID: "680e81a3-8ef0-4029-a6a0-625c24882954"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:55.000647 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.000621    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-bundle" (OuterVolumeSpecName: "bundle") pod "890b894a-830c-49a3-8c26-656d9a5a849f" (UID: "890b894a-830c-49a3-8c26-656d9a5a849f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:55.002102 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.002073    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680e81a3-8ef0-4029-a6a0-625c24882954-kube-api-access-68ptc" (OuterVolumeSpecName: "kube-api-access-68ptc") pod "680e81a3-8ef0-4029-a6a0-625c24882954" (UID: "680e81a3-8ef0-4029-a6a0-625c24882954"). InnerVolumeSpecName "kube-api-access-68ptc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:58:55.002626 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.002604    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/890b894a-830c-49a3-8c26-656d9a5a849f-kube-api-access-t9j6v" (OuterVolumeSpecName: "kube-api-access-t9j6v") pod "890b894a-830c-49a3-8c26-656d9a5a849f" (UID: "890b894a-830c-49a3-8c26-656d9a5a849f"). InnerVolumeSpecName "kube-api-access-t9j6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:58:55.002700 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.002657    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984b48f8-14d0-4162-966a-301606e89af7-kube-api-access-kdz9l" (OuterVolumeSpecName: "kube-api-access-kdz9l") pod "984b48f8-14d0-4162-966a-301606e89af7" (UID: "984b48f8-14d0-4162-966a-301606e89af7"). InnerVolumeSpecName "kube-api-access-kdz9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:58:55.006130 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.006106    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-util" (OuterVolumeSpecName: "util") pod "680e81a3-8ef0-4029-a6a0-625c24882954" (UID: "680e81a3-8ef0-4029-a6a0-625c24882954"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:55.006602 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.006578    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-util" (OuterVolumeSpecName: "util") pod "890b894a-830c-49a3-8c26-656d9a5a849f" (UID: "890b894a-830c-49a3-8c26-656d9a5a849f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:55.006816 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.006787    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-util" (OuterVolumeSpecName: "util") pod "984b48f8-14d0-4162-966a-301606e89af7" (UID: "984b48f8-14d0-4162-966a-301606e89af7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:58:55.101065 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101011    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101065 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101040    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101065 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101050    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdz9l\" (UniqueName: \"kubernetes.io/projected/984b48f8-14d0-4162-966a-301606e89af7-kube-api-access-kdz9l\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101065 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101058    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-util\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101065 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101066    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/890b894a-830c-49a3-8c26-656d9a5a849f-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101305 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101074    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68ptc\" (UniqueName: \"kubernetes.io/projected/680e81a3-8ef0-4029-a6a0-625c24882954-kube-api-access-68ptc\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101305 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101083    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/680e81a3-8ef0-4029-a6a0-625c24882954-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101305 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101092    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/984b48f8-14d0-4162-966a-301606e89af7-bundle\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.101305 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.101099    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9j6v\" (UniqueName: \"kubernetes.io/projected/890b894a-830c-49a3-8c26-656d9a5a849f-kube-api-access-t9j6v\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 11:58:55.769082 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.769050    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d"
Sep 10 11:58:55.769537 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.769090    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7rvr5d" event={"ID":"680e81a3-8ef0-4029-a6a0-625c24882954","Type":"ContainerDied","Data":"c5ef601d0355a20d2bdad82d18620b6e8f2d27f4b7597714ac8a8d829e1d910a"}
Sep 10 11:58:55.769537 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.769146    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5ef601d0355a20d2bdad82d18620b6e8f2d27f4b7597714ac8a8d829e1d910a"
Sep 10 11:58:55.771079 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.771052    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf" event={"ID":"890b894a-830c-49a3-8c26-656d9a5a849f","Type":"ContainerDied","Data":"5a5753a4f21e3b1bfc7f3babaae6d9e09e734db61d392794df08da0a88866476"}
Sep 10 11:58:55.771229 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.771093    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a5753a4f21e3b1bfc7f3babaae6d9e09e734db61d392794df08da0a88866476"
Sep 10 11:58:55.771229 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.771061    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ahwvtf"
Sep 10 11:58:55.773015 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.772999    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s"
Sep 10 11:58:55.773126 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.772993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6vsb2s" event={"ID":"984b48f8-14d0-4162-966a-301606e89af7","Type":"ContainerDied","Data":"1f257671f6e406354af8f764431f47a289169cad3765416a76b3a3b799d5536d"}
Sep 10 11:58:55.773126 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:55.773060    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f257671f6e406354af8f764431f47a289169cad3765416a76b3a3b799d5536d"
Sep 10 11:58:58.168866 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:58:58.168830    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:00.906481 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:00.906447    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:02.661753 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:02.661716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:02.763338 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:02.763306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:03.342840 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:03.342809    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:03.788468 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:03.788436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:04.011781 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:04.011752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:04.913905 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:04.913875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:05.110293 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:05.110258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:05.277068 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:05.277030    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:06.990416 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:06.990374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:07.327387 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:07.327305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:08.754685 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:08.754648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:10.726351 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:10.726319    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:10.837100 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:10.837064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:14.286746 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:14.286714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:14.373698 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:14.373666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:14.577884 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:14.577810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:18.561285 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:18.561244    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:18.657567 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:18.657533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:27.177788 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:27.177755    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:27.276454 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:27.276421    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:29.807333 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:29.807300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:29.917231 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:29.917197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:32.977384 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:32.977343    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:33.077942 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:33.077907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:38.191917 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:38.191877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:38.296066 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:38.296033    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:38.801661 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:38.801627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:38.916439 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:38.916400    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:39.379134 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:39.379099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:39.478789 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:39.478759    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:47.986148 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:47.986108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:48.085459 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:48.085406    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:48.558684 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:48.558652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:48.660616 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:48.660586    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:52.569341 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:52.569287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:52.669690 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:52.669658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:54.233142 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:54.233108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:54.339324 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:54.339283    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:58.848460 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:58.848415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:58.954700 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:58.954668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:59.023227 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:59.023199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:59.781791 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:59.781752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 11:59:59.885476 ip-10-0-129-41 kubenswrapper[2571]: I0910 11:59:59.885424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:10.751426 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:10.751391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:11.336228 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:11.336192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:15.063371 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:15.063335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:15.172818 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:15.172782    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:16.047510 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:16.047477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:23.235854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:23.235779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:26.657830 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:26.657795    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:27.557892 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:27.557854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:34.423581 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.423549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:34.467146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467109    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84489b7c8b-tvd4l"]
Sep 10 12:00:34.467407 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467395    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="pull"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467409    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="pull"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467417    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="util"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467422    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="util"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467430    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="util"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467436    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="util"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467442    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="extract"
Sep 10 12:00:34.467450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467447    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467455    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467461    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467466    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467471    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467479    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="pull"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467483    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="pull"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467488    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="pull"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467493    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="pull"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467500    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467504    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467512    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="pull"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467517    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="pull"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467524    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="util"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467529    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="util"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467538    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="util"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467542    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="util"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467585    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="680e81a3-8ef0-4029-a6a0-625c24882954" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467593    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f14afbc-663d-4e04-8ba8-3e8e2fac4ddc" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467599    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="890b894a-830c-49a3-8c26-656d9a5a849f" containerName="extract"
Sep 10 12:00:34.467663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.467605    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="984b48f8-14d0-4162-966a-301606e89af7" containerName="extract"
Sep 10 12:00:34.470434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.470418    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.473326 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.473297    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 12:00:34.473466 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.473329    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-smn6x\""
Sep 10 12:00:34.473466 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.473384    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 12:00:34.473609 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.473596    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 12:00:34.481138 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.481117    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84489b7c8b-tvd4l"]
Sep 10 12:00:34.506398 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.506362    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.506601 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.506576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prbfx\" (UniqueName: \"kubernetes.io/projected/ea43cb5a-ab4c-490f-beef-c017851dfe29-kube-api-access-prbfx\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.506728 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.506703    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-webhook-cert\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.520735 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.520703    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:34.607877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.607846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-webhook-cert\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.608070 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.607916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.608070 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.607941    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prbfx\" (UniqueName: \"kubernetes.io/projected/ea43cb5a-ab4c-490f-beef-c017851dfe29-kube-api-access-prbfx\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.608213 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:00:34.608061    2571 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 12:00:34.608213 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:00:34.608148    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs podName:ea43cb5a-ab4c-490f-beef-c017851dfe29 nodeName:}" failed. No retries permitted until 2026-09-10 12:00:35.108126659 +0000 UTC m=+545.711401562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs") pod "maas-controller-84489b7c8b-tvd4l" (UID: "ea43cb5a-ab4c-490f-beef-c017851dfe29") : secret "maas-controller-metrics-tls" not found
Sep 10 12:00:34.610526 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.610504    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-webhook-cert\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.617223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.617199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:34.618472 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.618450    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prbfx\" (UniqueName: \"kubernetes.io/projected/ea43cb5a-ab4c-490f-beef-c017851dfe29-kube-api-access-prbfx\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:34.690063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.690030    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-4kmsh"]
Sep 10 12:00:34.693313 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.693296    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.708532 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.708503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.708667 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.708574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxcmf\" (UniqueName: \"kubernetes.io/projected/951bbd11-849c-4cb7-8b44-559629964f98-kube-api-access-qxcmf\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.708667 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.708599    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-webhook-cert\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.710724 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.710699    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-4kmsh"]
Sep 10 12:00:34.810037 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.809998    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.810220 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.810053    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxcmf\" (UniqueName: \"kubernetes.io/projected/951bbd11-849c-4cb7-8b44-559629964f98-kube-api-access-qxcmf\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.810220 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.810070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-webhook-cert\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.810220 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:00:34.810165    2571 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 12:00:34.810332 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:00:34.810228    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs podName:951bbd11-849c-4cb7-8b44-559629964f98 nodeName:}" failed. No retries permitted until 2026-09-10 12:00:35.31021319 +0000 UTC m=+545.913488079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs") pod "maas-controller-6c88dd9b7f-4kmsh" (UID: "951bbd11-849c-4cb7-8b44-559629964f98") : secret "maas-controller-metrics-tls" not found
Sep 10 12:00:34.812451 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.812427    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-webhook-cert\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:34.819196 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:34.819138    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxcmf\" (UniqueName: \"kubernetes.io/projected/951bbd11-849c-4cb7-8b44-559629964f98-kube-api-access-qxcmf\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:35.045011 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.044927    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:35.112350 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.112316    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:35.114860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.114832    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs\") pod \"maas-controller-84489b7c8b-tvd4l\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") " pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:35.313368 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.313277    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:35.315629 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.315604    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs\") pod \"maas-controller-6c88dd9b7f-4kmsh\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") " pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:35.381798 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.381767    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:35.604051 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.603953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:35.654037 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.654005    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84489b7c8b-tvd4l"]
Sep 10 12:00:35.874976 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:35.874951    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-4kmsh"]
Sep 10 12:00:35.877292 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:00:35.877256    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951bbd11_849c_4cb7_8b44_559629964f98.slice/crio-2aa437a25e1af69bba09f407d32314ce8cddd9b9eed36dc42f333e491625c91f WatchSource:0}: Error finding container 2aa437a25e1af69bba09f407d32314ce8cddd9b9eed36dc42f333e491625c91f: Status 404 returned error can't find the container with id 2aa437a25e1af69bba09f407d32314ce8cddd9b9eed36dc42f333e491625c91f
Sep 10 12:00:36.233440 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:36.233399    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh" event={"ID":"951bbd11-849c-4cb7-8b44-559629964f98","Type":"ContainerStarted","Data":"2aa437a25e1af69bba09f407d32314ce8cddd9b9eed36dc42f333e491625c91f"}
Sep 10 12:00:36.234444 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:36.234421    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84489b7c8b-tvd4l" event={"ID":"ea43cb5a-ab4c-490f-beef-c017851dfe29","Type":"ContainerStarted","Data":"a6ae3b4c472ed8638e307124dcb0bfd48921aabb66677c154d3cfdb20067acc3"}
Sep 10 12:00:38.695236 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:38.695195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:38.794972 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:38.794880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:39.254407 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:39.254367    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh" event={"ID":"951bbd11-849c-4cb7-8b44-559629964f98","Type":"ContainerStarted","Data":"d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd"}
Sep 10 12:00:39.254616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:39.254433    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:39.255879 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:39.255854    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84489b7c8b-tvd4l" event={"ID":"ea43cb5a-ab4c-490f-beef-c017851dfe29","Type":"ContainerStarted","Data":"173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a"}
Sep 10 12:00:39.256029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:39.256013    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:39.286434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:39.286347    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh" podStartSLOduration=2.679996148 podStartE2EDuration="5.2863313s" totalImagesPullingTime="2.606335152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:00:39.283848889 +0000 UTC m=+549.887123823" watchObservedRunningTime="2026-09-10 12:00:39.2863313 +0000 UTC m=+549.889606244"
Sep 10 12:00:39.310099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:39.310047    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84489b7c8b-tvd4l" podStartSLOduration=2.470878802 podStartE2EDuration="5.310026939s" totalImagesPullingTime="2.839148137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:00:39.306859632 +0000 UTC m=+549.910134545" watchObservedRunningTime="2026-09-10 12:00:39.310026939 +0000 UTC m=+549.913301852"
Sep 10 12:00:41.586593 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:41.586551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:00:50.262457 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.262425    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:50.262890 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.262512    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:00:50.334351 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.334318    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84489b7c8b-tvd4l"]
Sep 10 12:00:50.334574 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.334544    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84489b7c8b-tvd4l" podUID="ea43cb5a-ab4c-490f-beef-c017851dfe29" containerName="manager" containerID="cri-o://173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a" gracePeriod=30
Sep 10 12:00:50.569414 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.569391    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:50.639324 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.639288    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prbfx\" (UniqueName: \"kubernetes.io/projected/ea43cb5a-ab4c-490f-beef-c017851dfe29-kube-api-access-prbfx\") pod \"ea43cb5a-ab4c-490f-beef-c017851dfe29\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") "
Sep 10 12:00:50.639493 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.639343    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-webhook-cert\") pod \"ea43cb5a-ab4c-490f-beef-c017851dfe29\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") "
Sep 10 12:00:50.639493 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.639364    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs\") pod \"ea43cb5a-ab4c-490f-beef-c017851dfe29\" (UID: \"ea43cb5a-ab4c-490f-beef-c017851dfe29\") "
Sep 10 12:00:50.641489 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.641452    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ea43cb5a-ab4c-490f-beef-c017851dfe29" (UID: "ea43cb5a-ab4c-490f-beef-c017851dfe29"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:00:50.641627 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.641517    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea43cb5a-ab4c-490f-beef-c017851dfe29" (UID: "ea43cb5a-ab4c-490f-beef-c017851dfe29"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:00:50.641627 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.641518    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea43cb5a-ab4c-490f-beef-c017851dfe29-kube-api-access-prbfx" (OuterVolumeSpecName: "kube-api-access-prbfx") pod "ea43cb5a-ab4c-490f-beef-c017851dfe29" (UID: "ea43cb5a-ab4c-490f-beef-c017851dfe29"). InnerVolumeSpecName "kube-api-access-prbfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:00:50.739998 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.739946    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prbfx\" (UniqueName: \"kubernetes.io/projected/ea43cb5a-ab4c-490f-beef-c017851dfe29-kube-api-access-prbfx\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:00:50.739998 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.739989    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-webhook-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:00:50.739998 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:50.740002    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea43cb5a-ab4c-490f-beef-c017851dfe29-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:00:51.314323 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.314286    2571 generic.go:356] "Generic (PLEG): container finished" podID="ea43cb5a-ab4c-490f-beef-c017851dfe29" containerID="173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a" exitCode=0
Sep 10 12:00:51.314793 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.314340    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84489b7c8b-tvd4l"
Sep 10 12:00:51.314793 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.314368    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84489b7c8b-tvd4l" event={"ID":"ea43cb5a-ab4c-490f-beef-c017851dfe29","Type":"ContainerDied","Data":"173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a"}
Sep 10 12:00:51.314793 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.314434    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84489b7c8b-tvd4l" event={"ID":"ea43cb5a-ab4c-490f-beef-c017851dfe29","Type":"ContainerDied","Data":"a6ae3b4c472ed8638e307124dcb0bfd48921aabb66677c154d3cfdb20067acc3"}
Sep 10 12:00:51.314793 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.314466    2571 scope.go:117] "RemoveContainer" containerID="173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a"
Sep 10 12:00:51.324058 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.324040    2571 scope.go:117] "RemoveContainer" containerID="173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a"
Sep 10 12:00:51.324335 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:00:51.324318    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a\": container with ID starting with 173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a not found: ID does not exist" containerID="173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a"
Sep 10 12:00:51.324387 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.324345    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a"} err="failed to get container status \"173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a\": rpc error: code = NotFound desc = could not find container \"173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a\": container with ID starting with 173fd0731af5b22f3b2a14026b84cb5b3ccad70699476ff0992f2dc73525929a not found: ID does not exist"
Sep 10 12:00:51.336790 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.336760    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84489b7c8b-tvd4l"]
Sep 10 12:00:51.345783 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.345755    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84489b7c8b-tvd4l"]
Sep 10 12:00:51.892247 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:00:51.892213    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea43cb5a-ab4c-490f-beef-c017851dfe29" path="/var/lib/kubelet/pods/ea43cb5a-ab4c-490f-beef-c017851dfe29/volumes"
Sep 10 12:01:14.263861 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:14.263822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:14.360191 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:14.360140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:15.678698 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.678660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:15.781657 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.781624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:15.799632 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.799601    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"]
Sep 10 12:01:15.799926 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.799914    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea43cb5a-ab4c-490f-beef-c017851dfe29" containerName="manager"
Sep 10 12:01:15.799986 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.799927    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea43cb5a-ab4c-490f-beef-c017851dfe29" containerName="manager"
Sep 10 12:01:15.800024 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.799987    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea43cb5a-ab4c-490f-beef-c017851dfe29" containerName="manager"
Sep 10 12:01:15.805836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.805817    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:15.808076 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.808048    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 12:01:15.808188 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.808172    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pjsm9\""
Sep 10 12:01:15.808951 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.808932    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 12:01:15.809033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.808938    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 12:01:15.809110 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.809083    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 12:01:15.815439 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.815417    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"]
Sep 10 12:01:15.875442 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.875415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:15.951010 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.950929    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-openshift-service-ca\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:15.951201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.951007    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-maas-api-tls\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:15.951201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.951055    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-metrics-certs\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:15.951201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:15.951078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lplqc\" (UniqueName: \"kubernetes.io/projected/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-kube-api-access-lplqc\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.052277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.052240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-openshift-service-ca\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.052434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.052293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-maas-api-tls\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.052434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.052320    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-metrics-certs\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.052434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.052339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lplqc\" (UniqueName: \"kubernetes.io/projected/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-kube-api-access-lplqc\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.053000 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.052978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-openshift-service-ca\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.054842 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.054818    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-maas-api-tls\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.054933 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.054850    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-metrics-certs\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.061805 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.061767    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lplqc\" (UniqueName: \"kubernetes.io/projected/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-kube-api-access-lplqc\") pod \"maas-api-5bb5f968cb-n6qnp\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") " pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.062052 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.062033    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:16.116996 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.116963    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:16.267838 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.267744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:16.361632 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.361597    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:16.390788 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.390758    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-scdzb"]
Sep 10 12:01:16.394902 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.394886    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.399975 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.399946    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-r8gmk\""
Sep 10 12:01:16.400086 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.400060    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 12:01:16.403335 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.403308    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"]
Sep 10 12:01:16.406982 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:01:16.406920    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46aecbb6_c487_47e3_88a6_c6f5adb9f2f8.slice/crio-167e452b6757eff985e79eb463aace463e232dd3d440296f10a60797cdc0a975 WatchSource:0}: Error finding container 167e452b6757eff985e79eb463aace463e232dd3d440296f10a60797cdc0a975: Status 404 returned error can't find the container with id 167e452b6757eff985e79eb463aace463e232dd3d440296f10a60797cdc0a975
Sep 10 12:01:16.408131 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.408111    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-scdzb"]
Sep 10 12:01:16.432578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.432542    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp" event={"ID":"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8","Type":"ContainerStarted","Data":"167e452b6757eff985e79eb463aace463e232dd3d440296f10a60797cdc0a975"}
Sep 10 12:01:16.556210 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.556098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sbnm\" (UniqueName: \"kubernetes.io/projected/b7713abc-e4fa-4b21-8453-a6fd6ad7a824-kube-api-access-2sbnm\") pod \"payload-processing-5b54b7fbb7-scdzb\" (UID: \"b7713abc-e4fa-4b21-8453-a6fd6ad7a824\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.556210 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.556172    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7713abc-e4fa-4b21-8453-a6fd6ad7a824-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-scdzb\" (UID: \"b7713abc-e4fa-4b21-8453-a6fd6ad7a824\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.656722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.656689    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7713abc-e4fa-4b21-8453-a6fd6ad7a824-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-scdzb\" (UID: \"b7713abc-e4fa-4b21-8453-a6fd6ad7a824\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.656891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.656765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sbnm\" (UniqueName: \"kubernetes.io/projected/b7713abc-e4fa-4b21-8453-a6fd6ad7a824-kube-api-access-2sbnm\") pod \"payload-processing-5b54b7fbb7-scdzb\" (UID: \"b7713abc-e4fa-4b21-8453-a6fd6ad7a824\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.657382 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.657360    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7713abc-e4fa-4b21-8453-a6fd6ad7a824-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-scdzb\" (UID: \"b7713abc-e4fa-4b21-8453-a6fd6ad7a824\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.672692 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.672662    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sbnm\" (UniqueName: \"kubernetes.io/projected/b7713abc-e4fa-4b21-8453-a6fd6ad7a824-kube-api-access-2sbnm\") pod \"payload-processing-5b54b7fbb7-scdzb\" (UID: \"b7713abc-e4fa-4b21-8453-a6fd6ad7a824\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.707662 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.707635    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:16.980334 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:16.980311    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-scdzb"]
Sep 10 12:01:16.982863 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:01:16.982832    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7713abc_e4fa_4b21_8453_a6fd6ad7a824.slice/crio-d7cc40dc6ac99c1b9e8c5c061bfaf5f13164044f022d920984b64e418a199c85 WatchSource:0}: Error finding container d7cc40dc6ac99c1b9e8c5c061bfaf5f13164044f022d920984b64e418a199c85: Status 404 returned error can't find the container with id d7cc40dc6ac99c1b9e8c5c061bfaf5f13164044f022d920984b64e418a199c85
Sep 10 12:01:17.438248 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:17.438217    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb" event={"ID":"b7713abc-e4fa-4b21-8453-a6fd6ad7a824","Type":"ContainerStarted","Data":"d7cc40dc6ac99c1b9e8c5c061bfaf5f13164044f022d920984b64e418a199c85"}
Sep 10 12:01:19.450368 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:19.450331    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb" event={"ID":"b7713abc-e4fa-4b21-8453-a6fd6ad7a824","Type":"ContainerStarted","Data":"c1b1777dafaa921f4c32b84a626470fd8ce3cac0fd5def4f21219ee8cec70e5d"}
Sep 10 12:01:19.450818 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:19.450410    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:19.451935 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:19.451914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp" event={"ID":"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8","Type":"ContainerStarted","Data":"ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a"}
Sep 10 12:01:19.452055 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:19.452041    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:19.469544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:19.469498    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb" podStartSLOduration=1.39065175 podStartE2EDuration="3.469480047s" totalImagesPullingTime="2.078828297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:01:19.468432061 +0000 UTC m=+590.071706985" watchObservedRunningTime="2026-09-10 12:01:19.469480047 +0000 UTC m=+590.072754958"
Sep 10 12:01:19.484540 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:19.484492    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp" podStartSLOduration=1.829154785 podStartE2EDuration="4.484476039s" totalImagesPullingTime="2.655321254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:01:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:01:19.483902919 +0000 UTC m=+590.087177834" watchObservedRunningTime="2026-09-10 12:01:19.484476039 +0000 UTC m=+590.087750978"
Sep 10 12:01:25.462372 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:25.462339    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:01:26.521809 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:26.521769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:26.615720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:26.615683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:29.941090 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:29.941056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:30.458752 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:30.458719    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-scdzb"
Sep 10 12:01:45.518202 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:45.518132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:46.214328 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:46.214294    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:49.589340 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:49.589306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:49.674454 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:49.674424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:54.053206 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:54.053110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:01:58.005984 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:01:58.005947    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:02:23.968045 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:23.968008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:02:34.280690 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:34.280655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:02:34.349205 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:34.349139    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:02:42.210589 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:42.210553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:02:54.368440 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.368404    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"]
Sep 10 12:02:54.372205 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.372172    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.376097 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.376069    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 12:02:54.376226 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.376100    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wfrs7\""
Sep 10 12:02:54.377053 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.377032    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 12:02:54.377337 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.377175    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 12:02:54.398655 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.398628    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"]
Sep 10 12:02:54.440437 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.440402    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:02:54.446544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.446524    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.446630 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.446551    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/459764d3-5850-4079-9aa9-a9bec7b196ab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.446630 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.446582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.446960 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.446648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.446960 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.446681    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.446960 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.446700    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhnsf\" (UniqueName: \"kubernetes.io/projected/459764d3-5850-4079-9aa9-a9bec7b196ab-kube-api-access-zhnsf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.547956 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.547916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.547956 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.547957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/459764d3-5850-4079-9aa9-a9bec7b196ab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548010    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548060    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548099    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548127    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhnsf\" (UniqueName: \"kubernetes.io/projected/459764d3-5850-4079-9aa9-a9bec7b196ab-kube-api-access-zhnsf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548418    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548549 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548524    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.548744 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.548722    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.550435 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.550412    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/459764d3-5850-4079-9aa9-a9bec7b196ab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.550857 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.550836    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/459764d3-5850-4079-9aa9-a9bec7b196ab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.559760 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.559728    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhnsf\" (UniqueName: \"kubernetes.io/projected/459764d3-5850-4079-9aa9-a9bec7b196ab-kube-api-access-zhnsf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6f292\" (UID: \"459764d3-5850-4079-9aa9-a9bec7b196ab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.682929 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.682902    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:02:54.957737 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.957713    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"]
Sep 10 12:02:54.959583 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:02:54.959561    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod459764d3_5850_4079_9aa9_a9bec7b196ab.slice/crio-9f42bf31652a1b8bbd03ab8cdbcaa5b508007d4359daa580de327c259cec16f8 WatchSource:0}: Error finding container 9f42bf31652a1b8bbd03ab8cdbcaa5b508007d4359daa580de327c259cec16f8: Status 404 returned error can't find the container with id 9f42bf31652a1b8bbd03ab8cdbcaa5b508007d4359daa580de327c259cec16f8
Sep 10 12:02:54.960859 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:54.960842    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:02:55.904718 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:55.904683    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292" event={"ID":"459764d3-5850-4079-9aa9-a9bec7b196ab","Type":"ContainerStarted","Data":"9f42bf31652a1b8bbd03ab8cdbcaa5b508007d4359daa580de327c259cec16f8"}
Sep 10 12:02:56.911084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:56.911048    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292" event={"ID":"459764d3-5850-4079-9aa9-a9bec7b196ab","Type":"ContainerStarted","Data":"4b58648c322616fdfa5244bef4759cf62f5d0be5d5d14e306292f2d21cb37496"}
Sep 10 12:02:56.949399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:02:56.949348    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292" podStartSLOduration=1.974390469 podStartE2EDuration="2.94933237s" totalImagesPullingTime="974.941901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:02:56.948735837 +0000 UTC m=+687.552010770" watchObservedRunningTime="2026-09-10 12:02:56.94933237 +0000 UTC m=+687.552607292"
Sep 10 12:03:02.627242 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:03:02.627126    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:03:04.683171 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:03:04.683113    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:03:04.683565 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:03:04.683211    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:03:04.695621 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:03:04.695598    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:03:04.958184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:03:04.958089    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6f292"
Sep 10 12:04:18.763031 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:18.762988    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:18.869289 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:18.869255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:19.999410 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:19.999373    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:20.103190 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:20.103138    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:36.148365 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.148330    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:36.263500 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.263468    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:36.472978 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.472942    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"]
Sep 10 12:04:36.476720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.476693    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.479416 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.479394    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-89d91a-openshift-default-dockercfg-j6wfr\""
Sep 10 12:04:36.493202 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.493173    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"]
Sep 10 12:04:36.501865 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.501832    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:36.512598 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512572    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-credential-socket\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512651    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-certs\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-podinfo\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512698    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-token\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512727    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vf25\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-kube-api-access-6vf25\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512974 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-socket\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512974 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77293c6d-96be-4e1d-ad18-b78993e11fdb-istiod-ca-cert\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512974 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512848    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-data\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.512974 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.512876    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-envoy\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.613614 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-data\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.613792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613619    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-envoy\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.613792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613658    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-credential-socket\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.613792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-certs\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.613792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613761    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-podinfo\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.613792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613782    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-token\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614068 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613810    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vf25\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-kube-api-access-6vf25\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614068 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613837    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-socket\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614068 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.613863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77293c6d-96be-4e1d-ad18-b78993e11fdb-istiod-ca-cert\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614068 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.614043    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-data\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614359 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.614203    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-certs\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614421 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.614398    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-socket\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614507 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.614479    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-credential-socket\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.614720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.614696    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77293c6d-96be-4e1d-ad18-b78993e11fdb-istiod-ca-cert\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.616141 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.616119    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-envoy\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.616389 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.616363    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-podinfo\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.624565 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.624537    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-token\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.625414 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.625392    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vf25\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-kube-api-access-6vf25\") pod \"e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") " pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:36.787957 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:36.787872    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:37.067518 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.067494    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"]
Sep 10 12:04:37.068936 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:04:37.068911    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77293c6d_96be_4e1d_ad18_b78993e11fdb.slice/crio-06b9a8daea5779831147c5aa21023d9b884d1a5d3653c6da9290afb3f67d7f97 WatchSource:0}: Error finding container 06b9a8daea5779831147c5aa21023d9b884d1a5d3653c6da9290afb3f67d7f97: Status 404 returned error can't find the container with id 06b9a8daea5779831147c5aa21023d9b884d1a5d3653c6da9290afb3f67d7f97
Sep 10 12:04:37.071003 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.070973    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:37.071058 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.071035    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:37.071101 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.071064    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:37.302545 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.302508    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:37.397379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.397293    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" event={"ID":"77293c6d-96be-4e1d-ad18-b78993e11fdb","Type":"ContainerStarted","Data":"8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"}
Sep 10 12:04:37.397379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.397373    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" event={"ID":"77293c6d-96be-4e1d-ad18-b78993e11fdb","Type":"ContainerStarted","Data":"06b9a8daea5779831147c5aa21023d9b884d1a5d3653c6da9290afb3f67d7f97"}
Sep 10 12:04:37.423978 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.423918    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podStartSLOduration=1.423903114 podStartE2EDuration="1.423903114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:04:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:04:37.422804271 +0000 UTC m=+788.026079208" watchObservedRunningTime="2026-09-10 12:04:37.423903114 +0000 UTC m=+788.027178026"
Sep 10 12:04:37.635094 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.635054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:37.788603 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.788571    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:37.807946 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:37.807917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:38.125916 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:38.125832    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:38.794216 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:38.794180    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:39.388033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:39.388002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:39.407256 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:39.407225    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:39.408234 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:39.408211    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:40.203836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.203801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:40.547501 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.547424    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"]
Sep 10 12:04:40.550666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.550647    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.554920 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.554894    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-89d91a-serving-cert\""
Sep 10 12:04:40.567823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.567783    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"]
Sep 10 12:04:40.606731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.606698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:40.649030 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.648998    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgwjc\" (UniqueName: \"kubernetes.io/projected/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-kube-api-access-zgwjc\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.649231 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.649046    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.649231 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.649066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-metrics-certs\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.649231 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.649184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.705829 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.705794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:40.750099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.750056    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.750099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.750095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-metrics-certs\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.750399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.750132    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.750399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.750205    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgwjc\" (UniqueName: \"kubernetes.io/projected/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-kube-api-access-zgwjc\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.751003 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.750977    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-openshift-service-ca\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.752673 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.752654    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-metrics-certs\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.752800 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.752780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-maas-api-tls\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.762898 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.762864    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgwjc\" (UniqueName: \"kubernetes.io/projected/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-kube-api-access-zgwjc\") pod \"maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:40.824910 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.824814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:40.861946 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:40.861911    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:41.142236 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:41.142202    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"]
Sep 10 12:04:41.144128 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:04:41.144098    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a21f5be_1500_4e8f_b073_4dfdb617f4f4.slice/crio-3bbf8d4d4961f2f51c016049f6b8520e015ffdbd5f77dfb31ec9e662c8390dd7 WatchSource:0}: Error finding container 3bbf8d4d4961f2f51c016049f6b8520e015ffdbd5f77dfb31ec9e662c8390dd7: Status 404 returned error can't find the container with id 3bbf8d4d4961f2f51c016049f6b8520e015ffdbd5f77dfb31ec9e662c8390dd7
Sep 10 12:04:41.236295 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:41.236259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:41.341482 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:41.341446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:41.418071 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:41.417972    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l" event={"ID":"0a21f5be-1500-4e8f-b073-4dfdb617f4f4","Type":"ContainerStarted","Data":"3bbf8d4d4961f2f51c016049f6b8520e015ffdbd5f77dfb31ec9e662c8390dd7"}
Sep 10 12:04:42.431364 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:42.431325    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l" event={"ID":"0a21f5be-1500-4e8f-b073-4dfdb617f4f4","Type":"ContainerStarted","Data":"ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6"}
Sep 10 12:04:42.431747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:42.431465    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:42.451636 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:42.451575    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l" podStartSLOduration=1.970601909 podStartE2EDuration="2.451559117s" totalImagesPullingTime="480.957208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:04:42.4500141 +0000 UTC m=+793.053289010" watchObservedRunningTime="2026-09-10 12:04:42.451559117 +0000 UTC m=+793.054834028"
Sep 10 12:04:44.048784 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:44.048752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:44.156399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:44.156357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:44.293421 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:44.293390    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:44.647480 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:44.647448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:44.838513 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:44.838479    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:47.700507 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:47.700476    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:47.789268 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:47.789236    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:48.191169 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.191124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:48.444059 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.443959    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:04:48.474455 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.474424    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"]
Sep 10 12:04:48.488175 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.484582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.496174 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.493329    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"]
Sep 10 12:04:48.496174 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.493568    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-7608bc-openshift-default-dockercfg-8n4lw\""
Sep 10 12:04:48.536540 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.536498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:48.624688 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624646    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-kube-api-access-d6hdv\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.624688 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624687    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-data\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.624932 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624720    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-socket\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.624932 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-credential-socket\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.624932 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624878    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-podinfo\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.624932 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624901    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fc15578-77f0-4ed8-932e-6a94d42e1411-istiod-ca-cert\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.625126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-certs\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.625126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.624998    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-token\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.625126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.625086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-envoy\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.652719 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.652648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:48.726096 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.725999    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-credential-socket\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726096 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-podinfo\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726096 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726078    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fc15578-77f0-4ed8-932e-6a94d42e1411-istiod-ca-cert\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-certs\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726198    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-token\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726243    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-envoy\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726273    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-kube-api-access-d6hdv\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-data\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726355    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-socket\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726517    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-credential-socket\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726577    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-certs\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.726714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726653    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-data\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.727058 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.726803    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-socket\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.727239 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.727216    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fc15578-77f0-4ed8-932e-6a94d42e1411-istiod-ca-cert\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.728810 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.728785    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-envoy\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.729025 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.729007    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-podinfo\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.734640 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.734611    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-token\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.735209 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.735185    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-kube-api-access-d6hdv\") pod \"e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") " pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.818545 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.818503    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:48.951127 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.951090    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"]
Sep 10 12:04:48.954908 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.954878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:48.958754 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.958729    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7a26e68a-serving-cert\""
Sep 10 12:04:48.972365 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:48.972272    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"]
Sep 10 12:04:49.003234 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.001691    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:49.028113 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.028078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a2b1103-a208-4be6-81b1-7d73064684c5-openshift-service-ca\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.028295 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.028126    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.028295 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.028177    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-metrics-certs\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.028295 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.028238    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjknx\" (UniqueName: \"kubernetes.io/projected/2a2b1103-a208-4be6-81b1-7d73064684c5-kube-api-access-sjknx\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.077429 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.077331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:49.110885 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.110852    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"]
Sep 10 12:04:49.113930 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:04:49.113895    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc15578_77f0_4ed8_932e_6a94d42e1411.slice/crio-08d8ac079a31b6ce60ff669022206334acf132b2c0519ee7a11a7e672f612185 WatchSource:0}: Error finding container 08d8ac079a31b6ce60ff669022206334acf132b2c0519ee7a11a7e672f612185: Status 404 returned error can't find the container with id 08d8ac079a31b6ce60ff669022206334acf132b2c0519ee7a11a7e672f612185
Sep 10 12:04:49.115969 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.115940    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:49.116065 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.116012    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:49.116065 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.116041    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:49.129357 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.129337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a2b1103-a208-4be6-81b1-7d73064684c5-openshift-service-ca\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.129449 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.129388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.129449 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.129413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-metrics-certs\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.129563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.129467    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjknx\" (UniqueName: \"kubernetes.io/projected/2a2b1103-a208-4be6-81b1-7d73064684c5-kube-api-access-sjknx\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.129563 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:04:49.129556    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-serving-cert: secret "maas-api-e2e-ait-7a26e68a-serving-cert" not found
Sep 10 12:04:49.129730 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:04:49.129616    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls podName:2a2b1103-a208-4be6-81b1-7d73064684c5 nodeName:}" failed. No retries permitted until 2026-09-10 12:04:49.629598457 +0000 UTC m=+800.232873349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls") pod "maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" (UID: "2a2b1103-a208-4be6-81b1-7d73064684c5") : secret "maas-api-e2e-ait-7a26e68a-serving-cert" not found
Sep 10 12:04:49.130046 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.130025    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a2b1103-a208-4be6-81b1-7d73064684c5-openshift-service-ca\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.134418 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.134396    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-metrics-certs\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.144992 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.144969    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjknx\" (UniqueName: \"kubernetes.io/projected/2a2b1103-a208-4be6-81b1-7d73064684c5-kube-api-access-sjknx\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.249768 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.249737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:49.471216 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.471174    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" event={"ID":"7fc15578-77f0-4ed8-932e-6a94d42e1411","Type":"ContainerStarted","Data":"5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"}
Sep 10 12:04:49.471373 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.471243    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" event={"ID":"7fc15578-77f0-4ed8-932e-6a94d42e1411","Type":"ContainerStarted","Data":"08d8ac079a31b6ce60ff669022206334acf132b2c0519ee7a11a7e672f612185"}
Sep 10 12:04:49.500371 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.500273    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podStartSLOduration=1.500246176 podStartE2EDuration="1.500246176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:04:49.497014282 +0000 UTC m=+800.100289194" watchObservedRunningTime="2026-09-10 12:04:49.500246176 +0000 UTC m=+800.103521090"
Sep 10 12:04:49.570742 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.570708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:49.634790 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.634757    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.637117 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.637090    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls\") pod \"maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:49.673699 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.673653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:49.819661 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.819561    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:49.866184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:49.866129    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:50.164763 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:50.164733    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"]
Sep 10 12:04:50.166321 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:04:50.166290    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2b1103_a208_4be6_81b1_7d73064684c5.slice/crio-f6b98bc7325f758af2c0ea1dfc3cbdb106974a519a6b81c4596b097564df39e9 WatchSource:0}: Error finding container f6b98bc7325f758af2c0ea1dfc3cbdb106974a519a6b81c4596b097564df39e9: Status 404 returned error can't find the container with id f6b98bc7325f758af2c0ea1dfc3cbdb106974a519a6b81c4596b097564df39e9
Sep 10 12:04:50.479882 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:50.479841    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" event={"ID":"2a2b1103-a208-4be6-81b1-7d73064684c5","Type":"ContainerStarted","Data":"f6b98bc7325f758af2c0ea1dfc3cbdb106974a519a6b81c4596b097564df39e9"}
Sep 10 12:04:50.824555 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:50.824470    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:51.488135 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:51.488098    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:51.492874 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:51.492631    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:04:51.702974 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:51.702932    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:51.802994 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:51.802954    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:52.390188 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.390132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:52.498245 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.498213    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" event={"ID":"2a2b1103-a208-4be6-81b1-7d73064684c5","Type":"ContainerStarted","Data":"965ea42f4189f9f4573db3fee40df3c1f313f45cc5b6668e35d056173cad7354"}
Sep 10 12:04:52.499001 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.498980    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:52.499602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.499571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:52.519976 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.519929    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" podStartSLOduration=2.917398158 podStartE2EDuration="4.51991355s" totalImagesPullingTime="1.602515392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:04:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:04:52.518388334 +0000 UTC m=+803.121663248" watchObservedRunningTime="2026-09-10 12:04:52.51991355 +0000 UTC m=+803.123188460"
Sep 10 12:04:52.863951 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.863918    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:52.967348 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:52.967312    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:53.300329 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:53.300287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:53.408201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:53.406539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:53.491622 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:53.491587    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:55.449325 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:55.449283    2571 patch_prober.go:28] interesting pod/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 12:04:55.449751 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:55.449359    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 12:04:56.322359 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:56.322317    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:56.422111 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:56.422076    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:57.529519 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:57.529485    2571 generic.go:356] "Generic (PLEG): container finished" podID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerID="8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702" exitCode=0
Sep 10 12:04:57.530208 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:57.530182    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" event={"ID":"77293c6d-96be-4e1d-ad18-b78993e11fdb","Type":"ContainerDied","Data":"8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"}
Sep 10 12:04:57.530762 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:57.530739    2571 scope.go:117] "RemoveContainer" containerID="8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"
Sep 10 12:04:57.532433 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:57.532390    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:57.532526 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:57.532478    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:57.532526 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:57.532520    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:04:58.354038 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:58.354001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:04:58.511038 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:58.511008    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:04:58.537067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:58.537031    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" event={"ID":"77293c6d-96be-4e1d-ad18-b78993e11fdb","Type":"ContainerStarted","Data":"c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d"}
Sep 10 12:04:58.789203 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:58.789165    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:04:58.789642 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:58.789532    2571 patch_prober.go:28] interesting pod/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 12:04:58.789826 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:58.789801    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 12:04:59.788507 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:59.788463    2571 patch_prober.go:28] interesting pod/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 12:04:59.788990 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:04:59.788549    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 12:05:00.267212 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:00.267177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:00.789181 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:00.789127    2571 patch_prober.go:28] interesting pod/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 12:05:00.789766 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:00.789730    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 12:05:01.691176 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:01.690973    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:01.790585 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:01.790544    2571 patch_prober.go:28] interesting pod/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 12:05:01.791177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:01.791117    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 12:05:01.871521 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:01.871440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:01.995131 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:01.995090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:02.793178 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:02.790555    2571 patch_prober.go:28] interesting pod/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Sep 10 12:05:02.793178 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:02.790622    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Sep 10 12:05:03.982312 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:03.982262    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:05:03.982838 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:03.982691    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:05:03.983720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:03.983691    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:05:04.678685 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:04.678655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:04.803831 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:04.803801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:05.158584 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:05.158503    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:05.887549 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:05.887517    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:08.243270 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:08.243179    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:08.315633 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:08.315599    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:08.425503 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:08.425453    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:08.548837 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:08.548749    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:08.722602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:08.722563    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:09.662288 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.662255    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"]
Sep 10 12:05:09.665731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.665713    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.668679 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.668658    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-425dc7-serving-cert\""
Sep 10 12:05:09.678454 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.678431    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"]
Sep 10 12:05:09.801702 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.801672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:09.825755 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.825716    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-metrics-certs\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.825911 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.825791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.825911 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.825816    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-maas-api-tls\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.825911 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.825839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9th87\" (UniqueName: \"kubernetes.io/projected/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-kube-api-access-9th87\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.926553 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.926471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-metrics-certs\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.926553 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.926534    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.926739 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.926559    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-maas-api-tls\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.926739 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.926577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9th87\" (UniqueName: \"kubernetes.io/projected/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-kube-api-access-9th87\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.927303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.927281    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.928967 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.928949    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-metrics-certs\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.929016 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.928992    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-maas-api-tls\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.936849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.936825    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9th87\" (UniqueName: \"kubernetes.io/projected/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-kube-api-access-9th87\") pod \"maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.975952 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.975924    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:09.995380 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:09.995350    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:10.257589 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.257562    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"]
Sep 10 12:05:10.259728 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:05:10.259698    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7338fa_25a3_40d2_a297_e90e3a4bb17f.slice/crio-25fb3f7acd38d49ef9e93e47dab34f5829e27b7435e43b3db1113c7c82b912fc WatchSource:0}: Error finding container 25fb3f7acd38d49ef9e93e47dab34f5829e27b7435e43b3db1113c7c82b912fc: Status 404 returned error can't find the container with id 25fb3f7acd38d49ef9e93e47dab34f5829e27b7435e43b3db1113c7c82b912fc
Sep 10 12:05:10.370613 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.370585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:10.448103 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.448063    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"]
Sep 10 12:05:10.452404 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.452381    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.454099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.454070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:10.455356 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.455328    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-425dc7\""
Sep 10 12:05:10.463807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.463784    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"]
Sep 10 12:05:10.533555 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.533479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdd88\" (UniqueName: \"kubernetes.io/projected/74cad910-d8b0-4281-87c4-36c91ae80d75-kube-api-access-kdd88\") pod \"payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.533555 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.533527    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cad910-d8b0-4281-87c4-36c91ae80d75-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.605469 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.605427    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj" event={"ID":"ae7338fa-25a3-40d2-a297-e90e3a4bb17f","Type":"ContainerStarted","Data":"25fb3f7acd38d49ef9e93e47dab34f5829e27b7435e43b3db1113c7c82b912fc"}
Sep 10 12:05:10.634830 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.634797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdd88\" (UniqueName: \"kubernetes.io/projected/74cad910-d8b0-4281-87c4-36c91ae80d75-kube-api-access-kdd88\") pod \"payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.634981 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.634852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cad910-d8b0-4281-87c4-36c91ae80d75-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.635649 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.635630    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cad910-d8b0-4281-87c4-36c91ae80d75-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.644141 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.644115    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdd88\" (UniqueName: \"kubernetes.io/projected/74cad910-d8b0-4281-87c4-36c91ae80d75-kube-api-access-kdd88\") pod \"payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:10.764640 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:10.764606    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:11.037102 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.037073    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"]
Sep 10 12:05:11.038963 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:05:11.038914    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74cad910_d8b0_4281_87c4_36c91ae80d75.slice/crio-f282a59912160d6a2185728ef62c5334443b8edaf73922dfab863cbd4d34e814 WatchSource:0}: Error finding container f282a59912160d6a2185728ef62c5334443b8edaf73922dfab863cbd4d34e814: Status 404 returned error can't find the container with id f282a59912160d6a2185728ef62c5334443b8edaf73922dfab863cbd4d34e814
Sep 10 12:05:11.614933 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.614897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj" event={"ID":"ae7338fa-25a3-40d2-a297-e90e3a4bb17f","Type":"ContainerStarted","Data":"131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384"}
Sep 10 12:05:11.615224 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.615200    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:11.616748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.616718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk" event={"ID":"74cad910-d8b0-4281-87c4-36c91ae80d75","Type":"ContainerStarted","Data":"d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b"}
Sep 10 12:05:11.616854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.616776    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:11.616854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.616804    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk" event={"ID":"74cad910-d8b0-4281-87c4-36c91ae80d75","Type":"ContainerStarted","Data":"f282a59912160d6a2185728ef62c5334443b8edaf73922dfab863cbd4d34e814"}
Sep 10 12:05:11.637253 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.637192    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj" podStartSLOduration=1.627398184 podStartE2EDuration="2.637179264s" totalImagesPullingTime="1.00978108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:05:11.634427925 +0000 UTC m=+822.237702838" watchObservedRunningTime="2026-09-10 12:05:11.637179264 +0000 UTC m=+822.240454176"
Sep 10 12:05:11.654083 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.654042    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk" podStartSLOduration=1.654030817 podStartE2EDuration="1.654030817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:05:11.65257453 +0000 UTC m=+822.255849454" watchObservedRunningTime="2026-09-10 12:05:11.654030817 +0000 UTC m=+822.257305728"
Sep 10 12:05:11.829867 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:11.829818    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:12.738823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:12.738782    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:14.239476 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:14.239442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:14.338396 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:14.338359    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:14.940368 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:14.940337    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:15.684548 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:15.684509    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:16.396364 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:16.396304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:16.678820 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:16.678786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:17.629719 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:17.629688    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:05:18.582284 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:18.582228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:22.529354 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:22.529309    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 12:05:22.529724 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:22.529387    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 12:05:22.625669 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:22.625638    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:05:24.690231 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:24.690193    2571 generic.go:356] "Generic (PLEG): container finished" podID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerID="5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f" exitCode=0
Sep 10 12:05:24.690777 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:24.690343    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" event={"ID":"7fc15578-77f0-4ed8-932e-6a94d42e1411","Type":"ContainerDied","Data":"5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"}
Sep 10 12:05:24.690847 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:24.690824    2571 scope.go:117] "RemoveContainer" containerID="5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"
Sep 10 12:05:24.692212 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:24.692182    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:05:24.692377 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:24.692266    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:05:24.692377 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:24.692314    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:05:25.701939 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:25.701902    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" event={"ID":"7fc15578-77f0-4ed8-932e-6a94d42e1411","Type":"ContainerStarted","Data":"59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59"}
Sep 10 12:05:25.819082 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:25.819039    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:05:25.819329 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:25.819300    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 10 12:05:25.819381 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:25.819358    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 10 12:05:26.819320 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:26.819279    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 10 12:05:26.819801 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:26.819357    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 10 12:05:27.819185 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:27.819111    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 10 12:05:27.819393 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:27.819241    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 10 12:05:28.819307 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:28.819267    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 10 12:05:28.819508 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:28.819333    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 10 12:05:29.503707 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.502814    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"]
Sep 10 12:05:29.503707 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.503065    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" podUID="2a2b1103-a208-4be6-81b1-7d73064684c5" containerName="maas-api" containerID="cri-o://965ea42f4189f9f4573db3fee40df3c1f313f45cc5b6668e35d056173cad7354" gracePeriod=30
Sep 10 12:05:29.533167 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.533114    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:29.622038 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.622005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:29.710100 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.710039    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:29.727623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.727594    2571 generic.go:356] "Generic (PLEG): container finished" podID="2a2b1103-a208-4be6-81b1-7d73064684c5" containerID="965ea42f4189f9f4573db3fee40df3c1f313f45cc5b6668e35d056173cad7354" exitCode=0
Sep 10 12:05:29.727768 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.727725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" event={"ID":"2a2b1103-a208-4be6-81b1-7d73064684c5","Type":"ContainerDied","Data":"965ea42f4189f9f4573db3fee40df3c1f313f45cc5b6668e35d056173cad7354"}
Sep 10 12:05:29.801219 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.800648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:29.811536 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.811513    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:05:29.819574 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.819548    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 10 12:05:29.819891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.819592    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 10 12:05:29.822904 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.822881    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjknx\" (UniqueName: \"kubernetes.io/projected/2a2b1103-a208-4be6-81b1-7d73064684c5-kube-api-access-sjknx\") pod \"2a2b1103-a208-4be6-81b1-7d73064684c5\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") "
Sep 10 12:05:29.822971 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.822936    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a2b1103-a208-4be6-81b1-7d73064684c5-openshift-service-ca\") pod \"2a2b1103-a208-4be6-81b1-7d73064684c5\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") "
Sep 10 12:05:29.823008 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.822991    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls\") pod \"2a2b1103-a208-4be6-81b1-7d73064684c5\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") "
Sep 10 12:05:29.823053 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.823010    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-metrics-certs\") pod \"2a2b1103-a208-4be6-81b1-7d73064684c5\" (UID: \"2a2b1103-a208-4be6-81b1-7d73064684c5\") "
Sep 10 12:05:29.823364 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.823336    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a2b1103-a208-4be6-81b1-7d73064684c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2a2b1103-a208-4be6-81b1-7d73064684c5" (UID: "2a2b1103-a208-4be6-81b1-7d73064684c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:05:29.825137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.825110    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2a2b1103-a208-4be6-81b1-7d73064684c5" (UID: "2a2b1103-a208-4be6-81b1-7d73064684c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:05:29.825255 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.825130    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2a2b1103-a208-4be6-81b1-7d73064684c5" (UID: "2a2b1103-a208-4be6-81b1-7d73064684c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:05:29.825255 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.825142    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2b1103-a208-4be6-81b1-7d73064684c5-kube-api-access-sjknx" (OuterVolumeSpecName: "kube-api-access-sjknx") pod "2a2b1103-a208-4be6-81b1-7d73064684c5" (UID: "2a2b1103-a208-4be6-81b1-7d73064684c5"). InnerVolumeSpecName "kube-api-access-sjknx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:05:29.874136 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.874100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:29.924386 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.924239    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjknx\" (UniqueName: \"kubernetes.io/projected/2a2b1103-a208-4be6-81b1-7d73064684c5-kube-api-access-sjknx\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:05:29.924386 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.924276    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a2b1103-a208-4be6-81b1-7d73064684c5-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:05:29.924386 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.924287    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:05:29.924386 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.924295    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a2b1103-a208-4be6-81b1-7d73064684c5-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:05:29.940633 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:29.940589    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:30.434797 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.434753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:30.526214 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.526167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:30.647389 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.647354    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:30.734333 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.734257    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"
Sep 10 12:05:30.734333 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.734285    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz" event={"ID":"2a2b1103-a208-4be6-81b1-7d73064684c5","Type":"ContainerDied","Data":"f6b98bc7325f758af2c0ea1dfc3cbdb106974a519a6b81c4596b097564df39e9"}
Sep 10 12:05:30.734577 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.734375    2571 scope.go:117] "RemoveContainer" containerID="965ea42f4189f9f4573db3fee40df3c1f313f45cc5b6668e35d056173cad7354"
Sep 10 12:05:30.754620 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.754595    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"]
Sep 10 12:05:30.758750 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.758728    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7a26e68a-dc48cdcb5-75qfz"]
Sep 10 12:05:30.818963 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.818915    2571 patch_prober.go:28] interesting pod/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 10 12:05:30.819126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.818983    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 10 12:05:30.995508 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:30.995419    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:31.342670 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:31.342569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:31.772856 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:31.772822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:31.822168 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:31.822122    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:05:31.822485 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:31.822465    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:05:31.823195 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:31.823170    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:05:31.894355 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:31.894227    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2b1103-a208-4be6-81b1-7d73064684c5" path="/var/lib/kubelet/pods/2a2b1103-a208-4be6-81b1-7d73064684c5/volumes"
Sep 10 12:05:33.463900 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:33.463864    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:34.104944 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.104915    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:34.204949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.204914    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"]
Sep 10 12:05:34.205201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.205118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:34.205392 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.205374    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2b1103-a208-4be6-81b1-7d73064684c5" containerName="maas-api"
Sep 10 12:05:34.205448 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.205394    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2b1103-a208-4be6-81b1-7d73064684c5" containerName="maas-api"
Sep 10 12:05:34.205493 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.205465    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2b1103-a208-4be6-81b1-7d73064684c5" containerName="maas-api"
Sep 10 12:05:34.208666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.208647    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.211200 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.211179    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-17efa977-serving-cert\""
Sep 10 12:05:34.222204 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.222178    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"]
Sep 10 12:05:34.262774 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.262745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-metrics-certs\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.262930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.262810    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.262930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.262862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964aceb8-0635-42aa-88a2-f166bee4552c-openshift-service-ca\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.262930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.262878    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhmch\" (UniqueName: \"kubernetes.io/projected/964aceb8-0635-42aa-88a2-f166bee4552c-kube-api-access-zhmch\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.345195 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.345137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:34.363648 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.363574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-metrics-certs\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.363795 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.363644    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.363795 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.363693    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964aceb8-0635-42aa-88a2-f166bee4552c-openshift-service-ca\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.363795 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.363717    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhmch\" (UniqueName: \"kubernetes.io/projected/964aceb8-0635-42aa-88a2-f166bee4552c-kube-api-access-zhmch\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.363963 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:05:34.363792    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-serving-cert: secret "maas-api-e2e-migrate-17efa977-serving-cert" not found
Sep 10 12:05:34.363963 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:05:34.363861    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls podName:964aceb8-0635-42aa-88a2-f166bee4552c nodeName:}" failed. No retries permitted until 2026-09-10 12:05:34.863840843 +0000 UTC m=+845.467115732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls") pod "maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" (UID: "964aceb8-0635-42aa-88a2-f166bee4552c") : secret "maas-api-e2e-migrate-17efa977-serving-cert" not found
Sep 10 12:05:34.364419 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.364401    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964aceb8-0635-42aa-88a2-f166bee4552c-openshift-service-ca\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.366000 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.365978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-metrics-certs\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.372782 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.372760    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhmch\" (UniqueName: \"kubernetes.io/projected/964aceb8-0635-42aa-88a2-f166bee4552c-kube-api-access-zhmch\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.631831 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.631747    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:34.864549 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.864518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:34.868809 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.868778    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.871138 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.871110    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls\") pod \"maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:34.959998 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:34.959961    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:05:35.048660 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.048625    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"]
Sep 10 12:05:35.051738 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.051709    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.055042 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.054856    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-17efa977\""
Sep 10 12:05:35.055042 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.054917    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-17efa977-dockercfg-8m4vm\""
Sep 10 12:05:35.063836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.063811    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"]
Sep 10 12:05:35.121838 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.121809    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:35.172491 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.172348    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65ce77fa-6034-40a6-8c84-a90c498dafb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.172491 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.172404    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r76fp\" (UniqueName: \"kubernetes.io/projected/65ce77fa-6034-40a6-8c84-a90c498dafb6-kube-api-access-r76fp\") pod \"payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.273699 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.273609    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65ce77fa-6034-40a6-8c84-a90c498dafb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.273699 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.273655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r76fp\" (UniqueName: \"kubernetes.io/projected/65ce77fa-6034-40a6-8c84-a90c498dafb6-kube-api-access-r76fp\") pod \"payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.274332 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.274309    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65ce77fa-6034-40a6-8c84-a90c498dafb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.282783 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.282759    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r76fp\" (UniqueName: \"kubernetes.io/projected/65ce77fa-6034-40a6-8c84-a90c498dafb6-kube-api-access-r76fp\") pod \"payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.366044 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.366014    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.396134 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.396102    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"]
Sep 10 12:05:35.397370 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:05:35.397344    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964aceb8_0635_42aa_88a2_f166bee4552c.slice/crio-97333313faafb2d1b2108a3b12ad1a58ff2e5316658e3cea3670b1d1263eac19 WatchSource:0}: Error finding container 97333313faafb2d1b2108a3b12ad1a58ff2e5316658e3cea3670b1d1263eac19: Status 404 returned error can't find the container with id 97333313faafb2d1b2108a3b12ad1a58ff2e5316658e3cea3670b1d1263eac19
Sep 10 12:05:35.659046 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.658964    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"]
Sep 10 12:05:35.664109 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:05:35.664075    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ce77fa_6034_40a6_8c84_a90c498dafb6.slice/crio-f431f5b8f3bbbfb5342157992646a2da3f6be95b3eb713f49dad7877ea38a8ae WatchSource:0}: Error finding container f431f5b8f3bbbfb5342157992646a2da3f6be95b3eb713f49dad7877ea38a8ae: Status 404 returned error can't find the container with id f431f5b8f3bbbfb5342157992646a2da3f6be95b3eb713f49dad7877ea38a8ae
Sep 10 12:05:35.769257 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.769222    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2" event={"ID":"65ce77fa-6034-40a6-8c84-a90c498dafb6","Type":"ContainerStarted","Data":"900eb67aa49789a800021152935c2cad666076656548ff29ec559b1fa34026a1"}
Sep 10 12:05:35.769461 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.769292    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:05:35.769461 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.769316    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2" event={"ID":"65ce77fa-6034-40a6-8c84-a90c498dafb6","Type":"ContainerStarted","Data":"f431f5b8f3bbbfb5342157992646a2da3f6be95b3eb713f49dad7877ea38a8ae"}
Sep 10 12:05:35.770569 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.770534    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" event={"ID":"964aceb8-0635-42aa-88a2-f166bee4552c","Type":"ContainerStarted","Data":"97333313faafb2d1b2108a3b12ad1a58ff2e5316658e3cea3670b1d1263eac19"}
Sep 10 12:05:35.788336 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:35.788283    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2" podStartSLOduration=0.788269102 podStartE2EDuration="788.269102ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:05:35.786481846 +0000 UTC m=+846.389756760" watchObservedRunningTime="2026-09-10 12:05:35.788269102 +0000 UTC m=+846.391544013"
Sep 10 12:05:36.777762 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:36.777725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" event={"ID":"964aceb8-0635-42aa-88a2-f166bee4552c","Type":"ContainerStarted","Data":"dd781b71c046d05c87e28c0ffe763bc9b095d0579b5a435097ed1ad0cee2d353"}
Sep 10 12:05:36.778249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:36.777839    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:36.795647 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:36.795596    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" podStartSLOduration=2.376460256 podStartE2EDuration="2.795578674s" totalImagesPullingTime="419.118418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:05:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:05:36.795232364 +0000 UTC m=+847.398507302" watchObservedRunningTime="2026-09-10 12:05:36.795578674 +0000 UTC m=+847.398853585"
Sep 10 12:05:42.788072 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:42.788039    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:05:46.780020 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:05:46.779980    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:06:05.716605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.716570    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"]
Sep 10 12:06:05.717027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.716788    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" podUID="964aceb8-0635-42aa-88a2-f166bee4552c" containerName="maas-api" containerID="cri-o://dd781b71c046d05c87e28c0ffe763bc9b095d0579b5a435097ed1ad0cee2d353" gracePeriod=30
Sep 10 12:06:05.751901 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.751840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:05.784686 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.784649    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"]
Sep 10 12:06:05.784929 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.784899    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2" podUID="65ce77fa-6034-40a6-8c84-a90c498dafb6" containerName="payload-pre-processing" containerID="cri-o://900eb67aa49789a800021152935c2cad666076656548ff29ec559b1fa34026a1" gracePeriod=30
Sep 10 12:06:05.933515 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.933459    2571 generic.go:356] "Generic (PLEG): container finished" podID="65ce77fa-6034-40a6-8c84-a90c498dafb6" containerID="900eb67aa49789a800021152935c2cad666076656548ff29ec559b1fa34026a1" exitCode=0
Sep 10 12:06:05.933929 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.933654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2" event={"ID":"65ce77fa-6034-40a6-8c84-a90c498dafb6","Type":"ContainerDied","Data":"900eb67aa49789a800021152935c2cad666076656548ff29ec559b1fa34026a1"}
Sep 10 12:06:05.935903 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.935879    2571 generic.go:356] "Generic (PLEG): container finished" podID="964aceb8-0635-42aa-88a2-f166bee4552c" containerID="dd781b71c046d05c87e28c0ffe763bc9b095d0579b5a435097ed1ad0cee2d353" exitCode=0
Sep 10 12:06:05.935999 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.935945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" event={"ID":"964aceb8-0635-42aa-88a2-f166bee4552c","Type":"ContainerDied","Data":"dd781b71c046d05c87e28c0ffe763bc9b095d0579b5a435097ed1ad0cee2d353"}
Sep 10 12:06:05.949066 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:05.949034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:06.033539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.033515    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:06:06.047409 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.047371    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:06.063810 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.063786    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:06:06.122562 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.122534    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:06.155796 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.155768    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65ce77fa-6034-40a6-8c84-a90c498dafb6-plugins-config-volume\") pod \"65ce77fa-6034-40a6-8c84-a90c498dafb6\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") "
Sep 10 12:06:06.155937 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.155828    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r76fp\" (UniqueName: \"kubernetes.io/projected/65ce77fa-6034-40a6-8c84-a90c498dafb6-kube-api-access-r76fp\") pod \"65ce77fa-6034-40a6-8c84-a90c498dafb6\" (UID: \"65ce77fa-6034-40a6-8c84-a90c498dafb6\") "
Sep 10 12:06:06.155937 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.155862    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhmch\" (UniqueName: \"kubernetes.io/projected/964aceb8-0635-42aa-88a2-f166bee4552c-kube-api-access-zhmch\") pod \"964aceb8-0635-42aa-88a2-f166bee4552c\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") "
Sep 10 12:06:06.155937 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.155893    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964aceb8-0635-42aa-88a2-f166bee4552c-openshift-service-ca\") pod \"964aceb8-0635-42aa-88a2-f166bee4552c\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") "
Sep 10 12:06:06.156103 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.155935    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-metrics-certs\") pod \"964aceb8-0635-42aa-88a2-f166bee4552c\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") "
Sep 10 12:06:06.156103 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.155962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls\") pod \"964aceb8-0635-42aa-88a2-f166bee4552c\" (UID: \"964aceb8-0635-42aa-88a2-f166bee4552c\") "
Sep 10 12:06:06.156251 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.156218    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65ce77fa-6034-40a6-8c84-a90c498dafb6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65ce77fa-6034-40a6-8c84-a90c498dafb6" (UID: "65ce77fa-6034-40a6-8c84-a90c498dafb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:06:06.156310 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.156256    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/964aceb8-0635-42aa-88a2-f166bee4552c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "964aceb8-0635-42aa-88a2-f166bee4552c" (UID: "964aceb8-0635-42aa-88a2-f166bee4552c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:06:06.158020 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.157980    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "964aceb8-0635-42aa-88a2-f166bee4552c" (UID: "964aceb8-0635-42aa-88a2-f166bee4552c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:06:06.158126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.158026    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "964aceb8-0635-42aa-88a2-f166bee4552c" (UID: "964aceb8-0635-42aa-88a2-f166bee4552c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:06:06.158126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.158081    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65ce77fa-6034-40a6-8c84-a90c498dafb6-kube-api-access-r76fp" (OuterVolumeSpecName: "kube-api-access-r76fp") pod "65ce77fa-6034-40a6-8c84-a90c498dafb6" (UID: "65ce77fa-6034-40a6-8c84-a90c498dafb6"). InnerVolumeSpecName "kube-api-access-r76fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:06:06.158126 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.158107    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964aceb8-0635-42aa-88a2-f166bee4552c-kube-api-access-zhmch" (OuterVolumeSpecName: "kube-api-access-zhmch") pod "964aceb8-0635-42aa-88a2-f166bee4552c" (UID: "964aceb8-0635-42aa-88a2-f166bee4552c"). InnerVolumeSpecName "kube-api-access-zhmch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:06:06.257517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.257434    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/964aceb8-0635-42aa-88a2-f166bee4552c-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:06.257517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.257459    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:06.257517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.257468    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/964aceb8-0635-42aa-88a2-f166bee4552c-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:06.257517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.257479    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65ce77fa-6034-40a6-8c84-a90c498dafb6-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:06.257517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.257489    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r76fp\" (UniqueName: \"kubernetes.io/projected/65ce77fa-6034-40a6-8c84-a90c498dafb6-kube-api-access-r76fp\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:06.257517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.257498    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhmch\" (UniqueName: \"kubernetes.io/projected/964aceb8-0635-42aa-88a2-f166bee4552c-kube-api-access-zhmch\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:06.553578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.553468    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:06.884523 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.884421    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:06.945133 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.944770    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2" event={"ID":"65ce77fa-6034-40a6-8c84-a90c498dafb6","Type":"ContainerDied","Data":"f431f5b8f3bbbfb5342157992646a2da3f6be95b3eb713f49dad7877ea38a8ae"}
Sep 10 12:06:06.945133 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.944840    2571 scope.go:117] "RemoveContainer" containerID="900eb67aa49789a800021152935c2cad666076656548ff29ec559b1fa34026a1"
Sep 10 12:06:06.945133 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.945003    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"
Sep 10 12:06:06.960348 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.960310    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh" event={"ID":"964aceb8-0635-42aa-88a2-f166bee4552c","Type":"ContainerDied","Data":"97333313faafb2d1b2108a3b12ad1a58ff2e5316658e3cea3670b1d1263eac19"}
Sep 10 12:06:06.960501 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.960342    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"
Sep 10 12:06:06.977280 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.977148    2571 scope.go:117] "RemoveContainer" containerID="dd781b71c046d05c87e28c0ffe763bc9b095d0579b5a435097ed1ad0cee2d353"
Sep 10 12:06:06.982061 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.982037    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"]
Sep 10 12:06:06.985866 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.985729    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-17efa977-69f58b9fd-q7nm2"]
Sep 10 12:06:07.001415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:06.997938    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"]
Sep 10 12:06:07.003352 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:07.003330    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17efa977-868c955f6c-c4pdh"]
Sep 10 12:06:07.311110 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:07.311069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:07.892334 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:07.892302    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65ce77fa-6034-40a6-8c84-a90c498dafb6" path="/var/lib/kubelet/pods/65ce77fa-6034-40a6-8c84-a90c498dafb6/volumes"
Sep 10 12:06:07.892720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:07.892707    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="964aceb8-0635-42aa-88a2-f166bee4552c" path="/var/lib/kubelet/pods/964aceb8-0635-42aa-88a2-f166bee4552c/volumes"
Sep 10 12:06:09.602860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:09.602826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:09.709164 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:09.709125    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:10.439613 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.439575    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"]
Sep 10 12:06:10.440134 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.440113    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="964aceb8-0635-42aa-88a2-f166bee4552c" containerName="maas-api"
Sep 10 12:06:10.440214 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.440138    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="964aceb8-0635-42aa-88a2-f166bee4552c" containerName="maas-api"
Sep 10 12:06:10.440214 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.440182    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65ce77fa-6034-40a6-8c84-a90c498dafb6" containerName="payload-pre-processing"
Sep 10 12:06:10.440214 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.440192    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="65ce77fa-6034-40a6-8c84-a90c498dafb6" containerName="payload-pre-processing"
Sep 10 12:06:10.440333 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.440319    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="65ce77fa-6034-40a6-8c84-a90c498dafb6" containerName="payload-pre-processing"
Sep 10 12:06:10.440369 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.440345    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="964aceb8-0635-42aa-88a2-f166bee4552c" containerName="maas-api"
Sep 10 12:06:10.445398 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.445374    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.448209 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.448186    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-eb1d9122-serving-cert\""
Sep 10 12:06:10.457843 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.457815    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"]
Sep 10 12:06:10.593794 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.593749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72ffg\" (UniqueName: \"kubernetes.io/projected/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-kube-api-access-72ffg\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.593995 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.593803    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-metrics-certs\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.593995 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.593897    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-openshift-service-ca\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.593995 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.593923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.694883 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.694797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-openshift-service-ca\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.694883 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.694836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.695420 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.694896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72ffg\" (UniqueName: \"kubernetes.io/projected/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-kube-api-access-72ffg\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.695420 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.694925    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-metrics-certs\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.695420 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:06:10.695038    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-serving-cert: secret "maas-api-e2e-ait-eb1d9122-serving-cert" not found
Sep 10 12:06:10.695420 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:06:10.695129    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls podName:8ad825eb-e2fe-4b00-82d5-7b57f70cf343 nodeName:}" failed. No retries permitted until 2026-09-10 12:06:11.195104375 +0000 UTC m=+881.798379277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls") pod "maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" (UID: "8ad825eb-e2fe-4b00-82d5-7b57f70cf343") : secret "maas-api-e2e-ait-eb1d9122-serving-cert" not found
Sep 10 12:06:10.695573 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.695494    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-openshift-service-ca\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.697289 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.697269    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-metrics-certs\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:10.704002 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:10.703975    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72ffg\" (UniqueName: \"kubernetes.io/projected/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-kube-api-access-72ffg\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:11.040410 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.040333    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"]
Sep 10 12:06:11.043579 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.043560    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.046204 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.046181    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-eb1d9122-dockercfg-ng8zh\""
Sep 10 12:06:11.046384 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.046368    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-eb1d9122\""
Sep 10 12:06:11.056816 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.056794    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"]
Sep 10 12:06:11.199896 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.199860    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l8nh\" (UniqueName: \"kubernetes.io/projected/b6599a9e-621f-4693-a167-e06f4e53dfcc-kube-api-access-5l8nh\") pod \"payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.200063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.199907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:11.200063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.199979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6599a9e-621f-4693-a167-e06f4e53dfcc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.202358 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.202334    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls\") pod \"maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:11.300982 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.300907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8nh\" (UniqueName: \"kubernetes.io/projected/b6599a9e-621f-4693-a167-e06f4e53dfcc-kube-api-access-5l8nh\") pod \"payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.300982 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.300960    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6599a9e-621f-4693-a167-e06f4e53dfcc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.301613 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.301595    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6599a9e-621f-4693-a167-e06f4e53dfcc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.310346 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.310324    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8nh\" (UniqueName: \"kubernetes.io/projected/b6599a9e-621f-4693-a167-e06f4e53dfcc-kube-api-access-5l8nh\") pod \"payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.350960 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.350935    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:11.353874 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.353856    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.357803 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.357788    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:11.634262 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.634231    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"]
Sep 10 12:06:11.634394 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:06:11.634347    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6599a9e_621f_4693_a167_e06f4e53dfcc.slice/crio-4ef6c81686ef311f668f9e9bc353f8588de22e0cdd2a7135e32e077b12e566ca WatchSource:0}: Error finding container 4ef6c81686ef311f668f9e9bc353f8588de22e0cdd2a7135e32e077b12e566ca: Status 404 returned error can't find the container with id 4ef6c81686ef311f668f9e9bc353f8588de22e0cdd2a7135e32e077b12e566ca
Sep 10 12:06:11.641744 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.641724    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"]
Sep 10 12:06:11.642784 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:06:11.642756    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad825eb_e2fe_4b00_82d5_7b57f70cf343.slice/crio-89eae3642e8ac68277934bfc60fd57902da74f1b9dc6e30c485b06d8847487f1 WatchSource:0}: Error finding container 89eae3642e8ac68277934bfc60fd57902da74f1b9dc6e30c485b06d8847487f1: Status 404 returned error can't find the container with id 89eae3642e8ac68277934bfc60fd57902da74f1b9dc6e30c485b06d8847487f1
Sep 10 12:06:11.794010 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.793982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:11.992613 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.992577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94" event={"ID":"b6599a9e-621f-4693-a167-e06f4e53dfcc","Type":"ContainerStarted","Data":"82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e"}
Sep 10 12:06:11.992804 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.992643    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:11.992804 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.992666    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94" event={"ID":"b6599a9e-621f-4693-a167-e06f4e53dfcc","Type":"ContainerStarted","Data":"4ef6c81686ef311f668f9e9bc353f8588de22e0cdd2a7135e32e077b12e566ca"}
Sep 10 12:06:11.994000 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:11.993971    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" event={"ID":"8ad825eb-e2fe-4b00-82d5-7b57f70cf343","Type":"ContainerStarted","Data":"89eae3642e8ac68277934bfc60fd57902da74f1b9dc6e30c485b06d8847487f1"}
Sep 10 12:06:12.016535 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:12.016483    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94" podStartSLOduration=1.016463451 podStartE2EDuration="1.016463451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:12.013533379 +0000 UTC m=+882.616808292" watchObservedRunningTime="2026-09-10 12:06:12.016463451 +0000 UTC m=+882.619738364"
Sep 10 12:06:13.001070 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:13.001028    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" event={"ID":"8ad825eb-e2fe-4b00-82d5-7b57f70cf343","Type":"ContainerStarted","Data":"608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb"}
Sep 10 12:06:13.001594 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:13.001261    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:13.020175 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:13.020117    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" podStartSLOduration=2.5973977379999997 podStartE2EDuration="3.020102524s" totalImagesPullingTime="422.704786ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:13.01902541 +0000 UTC m=+883.622300324" watchObservedRunningTime="2026-09-10 12:06:13.020102524 +0000 UTC m=+883.623377435"
Sep 10 12:06:19.011930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:19.011898    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:23.003698 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:23.003622    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:37.702730 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:37.702697    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:41.109509 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.109474    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:41.304339 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.304304    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"]
Sep 10 12:06:41.304668 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.304633    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" podUID="8ad825eb-e2fe-4b00-82d5-7b57f70cf343" containerName="maas-api" containerID="cri-o://608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb" gracePeriod=30
Sep 10 12:06:41.350248 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.350214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:41.373891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.373823    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"]
Sep 10 12:06:41.374116 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.374047    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94" podUID="b6599a9e-621f-4693-a167-e06f4e53dfcc" containerName="payload-pre-processing" containerID="cri-o://82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e" gracePeriod=30
Sep 10 12:06:41.451199 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.451167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:41.558205 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.557040    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:41.625605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.625541    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:41.630267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.630243    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:41.663554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.663527    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:41.773198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773143    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72ffg\" (UniqueName: \"kubernetes.io/projected/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-kube-api-access-72ffg\") pod \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") "
Sep 10 12:06:41.773359 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773204    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-openshift-service-ca\") pod \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") "
Sep 10 12:06:41.773359 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773306    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls\") pod \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") "
Sep 10 12:06:41.773359 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773342    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-metrics-certs\") pod \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\" (UID: \"8ad825eb-e2fe-4b00-82d5-7b57f70cf343\") "
Sep 10 12:06:41.773533 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773380    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l8nh\" (UniqueName: \"kubernetes.io/projected/b6599a9e-621f-4693-a167-e06f4e53dfcc-kube-api-access-5l8nh\") pod \"b6599a9e-621f-4693-a167-e06f4e53dfcc\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") "
Sep 10 12:06:41.773533 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773420    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6599a9e-621f-4693-a167-e06f4e53dfcc-plugins-config-volume\") pod \"b6599a9e-621f-4693-a167-e06f4e53dfcc\" (UID: \"b6599a9e-621f-4693-a167-e06f4e53dfcc\") "
Sep 10 12:06:41.773641 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773596    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ad825eb-e2fe-4b00-82d5-7b57f70cf343" (UID: "8ad825eb-e2fe-4b00-82d5-7b57f70cf343"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:06:41.773850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773760    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:41.773989 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.773963    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6599a9e-621f-4693-a167-e06f4e53dfcc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6599a9e-621f-4693-a167-e06f4e53dfcc" (UID: "b6599a9e-621f-4693-a167-e06f4e53dfcc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:06:41.775912 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.775881    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ad825eb-e2fe-4b00-82d5-7b57f70cf343" (UID: "8ad825eb-e2fe-4b00-82d5-7b57f70cf343"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:06:41.775912 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.775892    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-kube-api-access-72ffg" (OuterVolumeSpecName: "kube-api-access-72ffg") pod "8ad825eb-e2fe-4b00-82d5-7b57f70cf343" (UID: "8ad825eb-e2fe-4b00-82d5-7b57f70cf343"). InnerVolumeSpecName "kube-api-access-72ffg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:06:41.776072 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.775966    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8ad825eb-e2fe-4b00-82d5-7b57f70cf343" (UID: "8ad825eb-e2fe-4b00-82d5-7b57f70cf343"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:06:41.776072 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.775978    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6599a9e-621f-4693-a167-e06f4e53dfcc-kube-api-access-5l8nh" (OuterVolumeSpecName: "kube-api-access-5l8nh") pod "b6599a9e-621f-4693-a167-e06f4e53dfcc" (UID: "b6599a9e-621f-4693-a167-e06f4e53dfcc"). InnerVolumeSpecName "kube-api-access-5l8nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:06:41.874731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.874695    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:41.874731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.874729    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:41.874731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.874739    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l8nh\" (UniqueName: \"kubernetes.io/projected/b6599a9e-621f-4693-a167-e06f4e53dfcc-kube-api-access-5l8nh\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:41.874938 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.874750    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6599a9e-621f-4693-a167-e06f4e53dfcc-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:41.874938 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:41.874758    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72ffg\" (UniqueName: \"kubernetes.io/projected/8ad825eb-e2fe-4b00-82d5-7b57f70cf343-kube-api-access-72ffg\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:06:42.165722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.165631    2571 generic.go:356] "Generic (PLEG): container finished" podID="b6599a9e-621f-4693-a167-e06f4e53dfcc" containerID="82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e" exitCode=0
Sep 10 12:06:42.165722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.165682    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"
Sep 10 12:06:42.166243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.165720    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94" event={"ID":"b6599a9e-621f-4693-a167-e06f4e53dfcc","Type":"ContainerDied","Data":"82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e"}
Sep 10 12:06:42.166243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.165774    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94" event={"ID":"b6599a9e-621f-4693-a167-e06f4e53dfcc","Type":"ContainerDied","Data":"4ef6c81686ef311f668f9e9bc353f8588de22e0cdd2a7135e32e077b12e566ca"}
Sep 10 12:06:42.166243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.165805    2571 scope.go:117] "RemoveContainer" containerID="82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e"
Sep 10 12:06:42.167270 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.167242    2571 generic.go:356] "Generic (PLEG): container finished" podID="8ad825eb-e2fe-4b00-82d5-7b57f70cf343" containerID="608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb" exitCode=0
Sep 10 12:06:42.167406 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.167298    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"
Sep 10 12:06:42.167406 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.167304    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" event={"ID":"8ad825eb-e2fe-4b00-82d5-7b57f70cf343","Type":"ContainerDied","Data":"608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb"}
Sep 10 12:06:42.167406 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.167350    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl" event={"ID":"8ad825eb-e2fe-4b00-82d5-7b57f70cf343","Type":"ContainerDied","Data":"89eae3642e8ac68277934bfc60fd57902da74f1b9dc6e30c485b06d8847487f1"}
Sep 10 12:06:42.174598 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.174581    2571 scope.go:117] "RemoveContainer" containerID="82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e"
Sep 10 12:06:42.174902 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:06:42.174881    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e\": container with ID starting with 82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e not found: ID does not exist" containerID="82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e"
Sep 10 12:06:42.175018 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.174914    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e"} err="failed to get container status \"82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e\": rpc error: code = NotFound desc = could not find container \"82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e\": container with ID starting with 82c84ac2bee7713b8cc4e89d9a22985431aa978999fb19e74e9f7a849556cc0e not found: ID does not exist"
Sep 10 12:06:42.175018 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.174938    2571 scope.go:117] "RemoveContainer" containerID="608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb"
Sep 10 12:06:42.183121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.183091    2571 scope.go:117] "RemoveContainer" containerID="608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb"
Sep 10 12:06:42.183512 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:06:42.183488    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb\": container with ID starting with 608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb not found: ID does not exist" containerID="608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb"
Sep 10 12:06:42.183638 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.183518    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb"} err="failed to get container status \"608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb\": rpc error: code = NotFound desc = could not find container \"608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb\": container with ID starting with 608ed4ad4cf6266509192710714c82cf033f36a792f62074c435d5d0697292cb not found: ID does not exist"
Sep 10 12:06:42.185564 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.185544    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"]
Sep 10 12:06:42.188958 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.188940    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1d9122-54f5dd7c-x28bl"]
Sep 10 12:06:42.198326 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.198306    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"]
Sep 10 12:06:42.201292 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.201274    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb1d9122-77966795cd-9vv94"]
Sep 10 12:06:42.855833 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:42.855791    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:43.083601 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:43.083573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:43.351087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:43.351053    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:43.676891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:43.676807    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:43.832384 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:43.832351    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:43.893395 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:43.893366    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad825eb-e2fe-4b00-82d5-7b57f70cf343" path="/var/lib/kubelet/pods/8ad825eb-e2fe-4b00-82d5-7b57f70cf343/volumes"
Sep 10 12:06:43.893724 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:43.893711    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6599a9e-621f-4693-a167-e06f4e53dfcc" path="/var/lib/kubelet/pods/b6599a9e-621f-4693-a167-e06f4e53dfcc/volumes"
Sep 10 12:06:44.372895 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:44.372861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:46.431505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:46.431474    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:46.522449 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:46.522415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:47.601551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601521    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"]
Sep 10 12:06:47.601920 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601850    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6599a9e-621f-4693-a167-e06f4e53dfcc" containerName="payload-pre-processing"
Sep 10 12:06:47.601920 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601861    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6599a9e-621f-4693-a167-e06f4e53dfcc" containerName="payload-pre-processing"
Sep 10 12:06:47.601920 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601881    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad825eb-e2fe-4b00-82d5-7b57f70cf343" containerName="maas-api"
Sep 10 12:06:47.601920 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601886    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad825eb-e2fe-4b00-82d5-7b57f70cf343" containerName="maas-api"
Sep 10 12:06:47.602096 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601942    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6599a9e-621f-4693-a167-e06f4e53dfcc" containerName="payload-pre-processing"
Sep 10 12:06:47.602096 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.601953    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad825eb-e2fe-4b00-82d5-7b57f70cf343" containerName="maas-api"
Sep 10 12:06:47.606491 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.606472    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.610193 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.610144    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4d5fbdf0-serving-cert\""
Sep 10 12:06:47.615516 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.615492    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"]
Sep 10 12:06:47.651997 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.651971    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:47.725723 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.725693    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qktqm\" (UniqueName: \"kubernetes.io/projected/90656eaf-fc7b-42cc-966a-983dbe7ff125-kube-api-access-qktqm\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.725860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.725756    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-maas-api-tls\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.725860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.725777    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-metrics-certs\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.725860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.725844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90656eaf-fc7b-42cc-966a-983dbe7ff125-openshift-service-ca\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.738005 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.737976    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:47.826799 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.826764    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90656eaf-fc7b-42cc-966a-983dbe7ff125-openshift-service-ca\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.826799 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.826839    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qktqm\" (UniqueName: \"kubernetes.io/projected/90656eaf-fc7b-42cc-966a-983dbe7ff125-kube-api-access-qktqm\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.827276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.826883    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-maas-api-tls\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.827276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.826907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-metrics-certs\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.827781 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.827755    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90656eaf-fc7b-42cc-966a-983dbe7ff125-openshift-service-ca\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.829530 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.829506    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-metrics-certs\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.829618 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.829574    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-maas-api-tls\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.835936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.835910    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qktqm\" (UniqueName: \"kubernetes.io/projected/90656eaf-fc7b-42cc-966a-983dbe7ff125-kube-api-access-qktqm\") pod \"maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:47.837529 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.837507    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:47.919285 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:47.919220    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:48.223259 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.223234    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"]
Sep 10 12:06:48.225170 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:06:48.225119    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90656eaf_fc7b_42cc_966a_983dbe7ff125.slice/crio-3cfd082020ca97a80f4b7939a65de6ca9dde8dd06a6fbbbe10b52559838c7259 WatchSource:0}: Error finding container 3cfd082020ca97a80f4b7939a65de6ca9dde8dd06a6fbbbe10b52559838c7259: Status 404 returned error can't find the container with id 3cfd082020ca97a80f4b7939a65de6ca9dde8dd06a6fbbbe10b52559838c7259
Sep 10 12:06:48.580512 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.580426    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"]
Sep 10 12:06:48.585033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.585016    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.587539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.587513    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-4d5fbdf0\""
Sep 10 12:06:48.587681 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.587524    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-4d5fbdf0-dockercfg-5zd2s\""
Sep 10 12:06:48.594996 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.594968    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"]
Sep 10 12:06:48.733834 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.733803    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6c8z\" (UniqueName: \"kubernetes.io/projected/f909dc40-3f8d-4644-b8e0-f8484f347ced-kube-api-access-g6c8z\") pod \"payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.734245 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.733869    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f909dc40-3f8d-4644-b8e0-f8484f347ced-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.761177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.761136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:48.834478 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.834393    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f909dc40-3f8d-4644-b8e0-f8484f347ced-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.834624 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.834517    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6c8z\" (UniqueName: \"kubernetes.io/projected/f909dc40-3f8d-4644-b8e0-f8484f347ced-kube-api-access-g6c8z\") pod \"payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.835092 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.835071    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f909dc40-3f8d-4644-b8e0-f8484f347ced-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.844623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.844596    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6c8z\" (UniqueName: \"kubernetes.io/projected/f909dc40-3f8d-4644-b8e0-f8484f347ced-kube-api-access-g6c8z\") pod \"payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:48.897402 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:48.897362    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:49.175009 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.174973    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"]
Sep 10 12:06:49.175915 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:06:49.175893    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf909dc40_3f8d_4644_b8e0_f8484f347ced.slice/crio-23872807109f295fe738d159c6f2bffc7e6340d542944e692748f0d7e6f09790 WatchSource:0}: Error finding container 23872807109f295fe738d159c6f2bffc7e6340d542944e692748f0d7e6f09790: Status 404 returned error can't find the container with id 23872807109f295fe738d159c6f2bffc7e6340d542944e692748f0d7e6f09790
Sep 10 12:06:49.206063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.206033    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg" event={"ID":"90656eaf-fc7b-42cc-966a-983dbe7ff125","Type":"ContainerStarted","Data":"4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d"}
Sep 10 12:06:49.206189 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.206094    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg" event={"ID":"90656eaf-fc7b-42cc-966a-983dbe7ff125","Type":"ContainerStarted","Data":"3cfd082020ca97a80f4b7939a65de6ca9dde8dd06a6fbbbe10b52559838c7259"}
Sep 10 12:06:49.206189 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.206128    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:06:49.207400 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.207381    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk" event={"ID":"f909dc40-3f8d-4644-b8e0-f8484f347ced","Type":"ContainerStarted","Data":"23872807109f295fe738d159c6f2bffc7e6340d542944e692748f0d7e6f09790"}
Sep 10 12:06:49.230296 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.230256    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg" podStartSLOduration=1.806721485 podStartE2EDuration="2.230239912s" totalImagesPullingTime="423.518427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:49.226954136 +0000 UTC m=+919.830229047" watchObservedRunningTime="2026-09-10 12:06:49.230239912 +0000 UTC m=+919.833514822"
Sep 10 12:06:49.254874 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:49.254847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:06:50.215379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:50.215334    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk" event={"ID":"f909dc40-3f8d-4644-b8e0-f8484f347ced","Type":"ContainerStarted","Data":"96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69"}
Sep 10 12:06:50.215910 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:50.215437    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:06:50.235310 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:50.235262    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk" podStartSLOduration=2.235247447 podStartE2EDuration="2.235247447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:06:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:06:50.233293421 +0000 UTC m=+920.836568354" watchObservedRunningTime="2026-09-10 12:06:50.235247447 +0000 UTC m=+920.838522389"
Sep 10 12:06:55.222679 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:06:55.222649    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:07:01.223343 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:01.223311    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:07:15.400380 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:15.400344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:17.165221 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.165187    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"]
Sep 10 12:07:17.168787 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.168751    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.171370 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.171349    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-fd49b346-kserve-self-signed-certs\""
Sep 10 12:07:17.186418 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.186388    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"]
Sep 10 12:07:17.299519 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.299487    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tls-certs\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.299519 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.299529    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-dshm\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.299730 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.299647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tmp-dir\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.299790 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.299730    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-model-cache\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.299790 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.299762    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkhz2\" (UniqueName: \"kubernetes.io/projected/35fa68b1-1bef-4594-beed-56f1fe4b54eb-kube-api-access-wkhz2\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.299874 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.299800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-home\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401393 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401358    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tmp-dir\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401560 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401413    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-model-cache\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401560 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401439    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkhz2\" (UniqueName: \"kubernetes.io/projected/35fa68b1-1bef-4594-beed-56f1fe4b54eb-kube-api-access-wkhz2\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401560 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401466    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-home\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401560 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401559    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tls-certs\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401780 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401591    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-dshm\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401896 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401865    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tmp-dir\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401872    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-model-cache\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.401964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.401927    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-home\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.403723 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.403701    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-dshm\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.404020 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.403992    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tls-certs\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.409785 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.409762    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkhz2\" (UniqueName: \"kubernetes.io/projected/35fa68b1-1bef-4594-beed-56f1fe4b54eb-kube-api-access-wkhz2\") pod \"e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") " pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.479549 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.479516    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:17.755059 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:17.754986    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"]
Sep 10 12:07:17.757980 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:07:17.757950    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35fa68b1_1bef_4594_beed_56f1fe4b54eb.slice/crio-a442bf1f2aa7537f6deb0c3b65fe0baeb3c2cf9ed56952b3272d94904f785d8a WatchSource:0}: Error finding container a442bf1f2aa7537f6deb0c3b65fe0baeb3c2cf9ed56952b3272d94904f785d8a: Status 404 returned error can't find the container with id a442bf1f2aa7537f6deb0c3b65fe0baeb3c2cf9ed56952b3272d94904f785d8a
Sep 10 12:07:18.079541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.079458    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:18.181720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.181684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:18.368439 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.368339    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r" event={"ID":"35fa68b1-1bef-4594-beed-56f1fe4b54eb","Type":"ContainerStarted","Data":"bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55"}
Sep 10 12:07:18.368439 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.368384    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r" event={"ID":"35fa68b1-1bef-4594-beed-56f1fe4b54eb","Type":"ContainerStarted","Data":"a442bf1f2aa7537f6deb0c3b65fe0baeb3c2cf9ed56952b3272d94904f785d8a"}
Sep 10 12:07:18.388461 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.388415    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r" podStartSLOduration=1.388400546 podStartE2EDuration="1.388400546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:07:18.385571412 +0000 UTC m=+948.988846324" watchObservedRunningTime="2026-09-10 12:07:18.388400546 +0000 UTC m=+948.991675457"
Sep 10 12:07:18.672399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.672320    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"]
Sep 10 12:07:18.672562 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.672535    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg" podUID="90656eaf-fc7b-42cc-966a-983dbe7ff125" containerName="maas-api" containerID="cri-o://4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d" gracePeriod=30
Sep 10 12:07:18.695477 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.695446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:18.728445 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.728414    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"]
Sep 10 12:07:18.728667 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.728642    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk" podUID="f909dc40-3f8d-4644-b8e0-f8484f347ced" containerName="payload-pre-processing" containerID="cri-o://96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69" gracePeriod=30
Sep 10 12:07:18.793200 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.793167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:18.896286 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.895752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:18.980349 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.980322    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:07:18.996026 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:18.995982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:19.005676 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.005621    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:07:19.097289 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.097257    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:19.116886 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.116862    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90656eaf-fc7b-42cc-966a-983dbe7ff125-openshift-service-ca\") pod \"90656eaf-fc7b-42cc-966a-983dbe7ff125\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") "
Sep 10 12:07:19.117021 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.116926    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6c8z\" (UniqueName: \"kubernetes.io/projected/f909dc40-3f8d-4644-b8e0-f8484f347ced-kube-api-access-g6c8z\") pod \"f909dc40-3f8d-4644-b8e0-f8484f347ced\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") "
Sep 10 12:07:19.117021 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.116948    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qktqm\" (UniqueName: \"kubernetes.io/projected/90656eaf-fc7b-42cc-966a-983dbe7ff125-kube-api-access-qktqm\") pod \"90656eaf-fc7b-42cc-966a-983dbe7ff125\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") "
Sep 10 12:07:19.117141 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117037    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f909dc40-3f8d-4644-b8e0-f8484f347ced-plugins-config-volume\") pod \"f909dc40-3f8d-4644-b8e0-f8484f347ced\" (UID: \"f909dc40-3f8d-4644-b8e0-f8484f347ced\") "
Sep 10 12:07:19.117456 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117132    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-maas-api-tls\") pod \"90656eaf-fc7b-42cc-966a-983dbe7ff125\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") "
Sep 10 12:07:19.117456 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117416    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-metrics-certs\") pod \"90656eaf-fc7b-42cc-966a-983dbe7ff125\" (UID: \"90656eaf-fc7b-42cc-966a-983dbe7ff125\") "
Sep 10 12:07:19.117594 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117340    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90656eaf-fc7b-42cc-966a-983dbe7ff125-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "90656eaf-fc7b-42cc-966a-983dbe7ff125" (UID: "90656eaf-fc7b-42cc-966a-983dbe7ff125"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:07:19.117594 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117425    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f909dc40-3f8d-4644-b8e0-f8484f347ced-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f909dc40-3f8d-4644-b8e0-f8484f347ced" (UID: "f909dc40-3f8d-4644-b8e0-f8484f347ced"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:07:19.117711 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117702    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90656eaf-fc7b-42cc-966a-983dbe7ff125-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:19.117770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.117715    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f909dc40-3f8d-4644-b8e0-f8484f347ced-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:19.119209 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.119183    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f909dc40-3f8d-4644-b8e0-f8484f347ced-kube-api-access-g6c8z" (OuterVolumeSpecName: "kube-api-access-g6c8z") pod "f909dc40-3f8d-4644-b8e0-f8484f347ced" (UID: "f909dc40-3f8d-4644-b8e0-f8484f347ced"). InnerVolumeSpecName "kube-api-access-g6c8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:07:19.119403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.119379    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "90656eaf-fc7b-42cc-966a-983dbe7ff125" (UID: "90656eaf-fc7b-42cc-966a-983dbe7ff125"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:07:19.119532 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.119511    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "90656eaf-fc7b-42cc-966a-983dbe7ff125" (UID: "90656eaf-fc7b-42cc-966a-983dbe7ff125"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:07:19.119605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.119589    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90656eaf-fc7b-42cc-966a-983dbe7ff125-kube-api-access-qktqm" (OuterVolumeSpecName: "kube-api-access-qktqm") pod "90656eaf-fc7b-42cc-966a-983dbe7ff125" (UID: "90656eaf-fc7b-42cc-966a-983dbe7ff125"). InnerVolumeSpecName "kube-api-access-qktqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:07:19.218804 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.218721    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6c8z\" (UniqueName: \"kubernetes.io/projected/f909dc40-3f8d-4644-b8e0-f8484f347ced-kube-api-access-g6c8z\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:19.218804 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.218749    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qktqm\" (UniqueName: \"kubernetes.io/projected/90656eaf-fc7b-42cc-966a-983dbe7ff125-kube-api-access-qktqm\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:19.218804 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.218758    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:19.218804 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.218766    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90656eaf-fc7b-42cc-966a-983dbe7ff125-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:19.374571 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.374540    2571 generic.go:356] "Generic (PLEG): container finished" podID="90656eaf-fc7b-42cc-966a-983dbe7ff125" containerID="4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d" exitCode=0
Sep 10 12:07:19.374756 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.374584    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"
Sep 10 12:07:19.374756 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.374625    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg" event={"ID":"90656eaf-fc7b-42cc-966a-983dbe7ff125","Type":"ContainerDied","Data":"4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d"}
Sep 10 12:07:19.374756 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.374676    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg" event={"ID":"90656eaf-fc7b-42cc-966a-983dbe7ff125","Type":"ContainerDied","Data":"3cfd082020ca97a80f4b7939a65de6ca9dde8dd06a6fbbbe10b52559838c7259"}
Sep 10 12:07:19.374756 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.374704    2571 scope.go:117] "RemoveContainer" containerID="4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d"
Sep 10 12:07:19.376059 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.376022    2571 generic.go:356] "Generic (PLEG): container finished" podID="f909dc40-3f8d-4644-b8e0-f8484f347ced" containerID="96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69" exitCode=0
Sep 10 12:07:19.376186 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.376069    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"
Sep 10 12:07:19.376186 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.376079    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk" event={"ID":"f909dc40-3f8d-4644-b8e0-f8484f347ced","Type":"ContainerDied","Data":"96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69"}
Sep 10 12:07:19.376186 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.376129    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk" event={"ID":"f909dc40-3f8d-4644-b8e0-f8484f347ced","Type":"ContainerDied","Data":"23872807109f295fe738d159c6f2bffc7e6340d542944e692748f0d7e6f09790"}
Sep 10 12:07:19.385737 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.385718    2571 scope.go:117] "RemoveContainer" containerID="4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d"
Sep 10 12:07:19.386122 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:07:19.386103    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d\": container with ID starting with 4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d not found: ID does not exist" containerID="4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d"
Sep 10 12:07:19.386229 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.386130    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d"} err="failed to get container status \"4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d\": rpc error: code = NotFound desc = could not find container \"4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d\": container with ID starting with 4037887a42e6e765c31e4a2cc4ebb8fc22b66f0277db74965c298d00587afb3d not found: ID does not exist"
Sep 10 12:07:19.386229 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.386146    2571 scope.go:117] "RemoveContainer" containerID="96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69"
Sep 10 12:07:19.394374 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.394359    2571 scope.go:117] "RemoveContainer" containerID="96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69"
Sep 10 12:07:19.394593 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:07:19.394578    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69\": container with ID starting with 96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69 not found: ID does not exist" containerID="96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69"
Sep 10 12:07:19.394640 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.394598    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69"} err="failed to get container status \"96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69\": rpc error: code = NotFound desc = could not find container \"96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69\": container with ID starting with 96b833715abf147afbe3871ba74193e626b4b6b396274f6051ce2d4c206cdb69 not found: ID does not exist"
Sep 10 12:07:19.402288 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.402268    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"]
Sep 10 12:07:19.407132 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.407111    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4d5fbdf0-797c5f6577-8jqgg"]
Sep 10 12:07:19.418926 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.418907    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"]
Sep 10 12:07:19.424056 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.424035    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-4d5fbdf0-7d6bc67f54-sggfk"]
Sep 10 12:07:19.769516 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.769478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:19.865446 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.865409    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:19.893568 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.893528    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90656eaf-fc7b-42cc-966a-983dbe7ff125" path="/var/lib/kubelet/pods/90656eaf-fc7b-42cc-966a-983dbe7ff125/volumes"
Sep 10 12:07:19.894018 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.893997    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f909dc40-3f8d-4644-b8e0-f8484f347ced" path="/var/lib/kubelet/pods/f909dc40-3f8d-4644-b8e0-f8484f347ced/volumes"
Sep 10 12:07:19.965337 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:19.965307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:21.400300 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:21.400263    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:25.474966 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:25.474910    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:26.354366 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:26.354336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:26.441678 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:26.441644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:26.542023 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:26.541985    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:26.608218 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:26.608128    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:26.939097 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:26.939051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:27.479940 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:27.479904    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:27.480124 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:27.479957    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:27.492602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:27.492570    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:28.437931 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:28.437895    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:29.494287 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:29.494255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:31.973378 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:31.973343    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:32.977348 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:32.977316    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:35.553292 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:35.553261    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:36.270846 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:36.270810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:37.139646 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:37.139605    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"]
Sep 10 12:07:37.159083 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:37.159052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:47.907749 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:47.907716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:47.983249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:47.983214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:50.479060 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:50.479024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:07:52.159345 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.159232    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r" podUID="35fa68b1-1bef-4594-beed-56f1fe4b54eb" containerName="main" containerID="cri-o://bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55" gracePeriod=45
Sep 10 12:07:52.397107 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.397080    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:52.514802 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.514767    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-home\") pod \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") "
Sep 10 12:07:52.514802 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.514806    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tls-certs\") pod \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") "
Sep 10 12:07:52.515064 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.514837    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-model-cache\") pod \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") "
Sep 10 12:07:52.515064 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.514853    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tmp-dir\") pod \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") "
Sep 10 12:07:52.515064 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.514875    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-dshm\") pod \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") "
Sep 10 12:07:52.515064 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.514916    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkhz2\" (UniqueName: \"kubernetes.io/projected/35fa68b1-1bef-4594-beed-56f1fe4b54eb-kube-api-access-wkhz2\") pod \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\" (UID: \"35fa68b1-1bef-4594-beed-56f1fe4b54eb\") "
Sep 10 12:07:52.515277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.515079    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-home" (OuterVolumeSpecName: "home") pod "35fa68b1-1bef-4594-beed-56f1fe4b54eb" (UID: "35fa68b1-1bef-4594-beed-56f1fe4b54eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:07:52.515277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.515126    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35fa68b1-1bef-4594-beed-56f1fe4b54eb" (UID: "35fa68b1-1bef-4594-beed-56f1fe4b54eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:07:52.515277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.515101    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "35fa68b1-1bef-4594-beed-56f1fe4b54eb" (UID: "35fa68b1-1bef-4594-beed-56f1fe4b54eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:07:52.517078 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.517046    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-dshm" (OuterVolumeSpecName: "dshm") pod "35fa68b1-1bef-4594-beed-56f1fe4b54eb" (UID: "35fa68b1-1bef-4594-beed-56f1fe4b54eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:07:52.517201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.517098    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35fa68b1-1bef-4594-beed-56f1fe4b54eb-kube-api-access-wkhz2" (OuterVolumeSpecName: "kube-api-access-wkhz2") pod "35fa68b1-1bef-4594-beed-56f1fe4b54eb" (UID: "35fa68b1-1bef-4594-beed-56f1fe4b54eb"). InnerVolumeSpecName "kube-api-access-wkhz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:07:52.517201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.517144    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35fa68b1-1bef-4594-beed-56f1fe4b54eb" (UID: "35fa68b1-1bef-4594-beed-56f1fe4b54eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:07:52.554974 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.554944    2571 generic.go:356] "Generic (PLEG): container finished" podID="35fa68b1-1bef-4594-beed-56f1fe4b54eb" containerID="bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55" exitCode=0
Sep 10 12:07:52.555072 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.554987    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"
Sep 10 12:07:52.555072 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.555023    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r" event={"ID":"35fa68b1-1bef-4594-beed-56f1fe4b54eb","Type":"ContainerDied","Data":"bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55"}
Sep 10 12:07:52.555072 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.555066    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r" event={"ID":"35fa68b1-1bef-4594-beed-56f1fe4b54eb","Type":"ContainerDied","Data":"a442bf1f2aa7537f6deb0c3b65fe0baeb3c2cf9ed56952b3272d94904f785d8a"}
Sep 10 12:07:52.555189 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.555082    2571 scope.go:117] "RemoveContainer" containerID="bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55"
Sep 10 12:07:52.563650 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.563630    2571 scope.go:117] "RemoveContainer" containerID="bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55"
Sep 10 12:07:52.563921 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:07:52.563904    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55\": container with ID starting with bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55 not found: ID does not exist" containerID="bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55"
Sep 10 12:07:52.563995 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.563926    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55"} err="failed to get container status \"bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55\": rpc error: code = NotFound desc = could not find container \"bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55\": container with ID starting with bbc298ebdfb394f3ebf72de51c0c8834b485adebcaf40a422d59ff2144a7db55 not found: ID does not exist"
Sep 10 12:07:52.577275 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.577252    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"]
Sep 10 12:07:52.580638 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.580616    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-fd49b346-kserve-756f594f87-dmm5r"]
Sep 10 12:07:52.616433 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.616410    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-home\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:52.616433 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.616430    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tls-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:52.616578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.616445    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-model-cache\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:52.616578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.616475    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-tmp-dir\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:52.616578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.616487    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35fa68b1-1bef-4594-beed-56f1fe4b54eb-dshm\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:52.616578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:52.616501    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkhz2\" (UniqueName: \"kubernetes.io/projected/35fa68b1-1bef-4594-beed-56f1fe4b54eb-kube-api-access-wkhz2\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:07:53.892601 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:53.892558    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35fa68b1-1bef-4594-beed-56f1fe4b54eb" path="/var/lib/kubelet/pods/35fa68b1-1bef-4594-beed-56f1fe4b54eb/volumes"
Sep 10 12:07:54.503541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:07:54.503507    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:00.530276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:00.530244    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:00.670663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:00.670634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:00.754945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:00.754915    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:09.359267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:09.359234    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:09.523321 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:09.523289    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:09.723117 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:09.723080    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"]
Sep 10 12:08:09.723331 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:09.723309    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy" containerID="cri-o://59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59" gracePeriod=30
Sep 10 12:08:09.780723 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:09.780697    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:10.176007 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:10.175929    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:10.510731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:10.510699    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:10.648130 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:10.648094    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:13.526218 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:13.526183    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:13.622308 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:13.622248    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:14.698848 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:14.698813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:14.800459 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:14.800426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:14.898352 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:14.898314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:14.975572 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:14.975545    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:08:15.115805 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115775    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-certs\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.115805 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115809    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-token\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115847    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-envoy\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115884    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-credential-socket\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115915    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-data\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115942    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-socket\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.115977    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-kube-api-access-d6hdv\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116000    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-podinfo\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116057    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fc15578-77f0-4ed8-932e-6a94d42e1411-istiod-ca-cert\") pod \"7fc15578-77f0-4ed8-932e-6a94d42e1411\" (UID: \"7fc15578-77f0-4ed8-932e-6a94d42e1411\") "
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116080    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116265    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116379    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-data" (OuterVolumeSpecName: "istio-data") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116406    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116411    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:08:15.116438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116426    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-credential-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.116783 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.116594    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fc15578-77f0-4ed8-932e-6a94d42e1411-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:08:15.118168 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.118128    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:08:15.118274 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.118232    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-kube-api-access-d6hdv" (OuterVolumeSpecName: "kube-api-access-d6hdv") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "kube-api-access-d6hdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:08:15.118314 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.118268    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 12:08:15.118314 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.118302    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-token" (OuterVolumeSpecName: "istio-token") pod "7fc15578-77f0-4ed8-932e-6a94d42e1411" (UID: "7fc15578-77f0-4ed8-932e-6a94d42e1411"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:08:15.195855 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.195824    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:15.217267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217195    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-data\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.217267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217221    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-workload-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.217267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217234    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6hdv\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-kube-api-access-d6hdv\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.217267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217248    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-podinfo\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.217267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217259    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fc15578-77f0-4ed8-932e-6a94d42e1411-istiod-ca-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.217267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217271    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-token\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.217555 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.217282    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fc15578-77f0-4ed8-932e-6a94d42e1411-istio-envoy\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:08:15.294888 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.294854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:15.313439 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313408    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"]
Sep 10 12:08:15.313794 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313776    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90656eaf-fc7b-42cc-966a-983dbe7ff125" containerName="maas-api"
Sep 10 12:08:15.313877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313797    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="90656eaf-fc7b-42cc-966a-983dbe7ff125" containerName="maas-api"
Sep 10 12:08:15.313877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313819    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f909dc40-3f8d-4644-b8e0-f8484f347ced" containerName="payload-pre-processing"
Sep 10 12:08:15.313877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313828    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f909dc40-3f8d-4644-b8e0-f8484f347ced" containerName="payload-pre-processing"
Sep 10 12:08:15.313877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313856    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy"
Sep 10 12:08:15.313877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313865    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy"
Sep 10 12:08:15.313877 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313877    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fa68b1-1bef-4594-beed-56f1fe4b54eb" containerName="main"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313885    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fa68b1-1bef-4594-beed-56f1fe4b54eb" containerName="main"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313895    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313903    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.313989    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.314002    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="90656eaf-fc7b-42cc-966a-983dbe7ff125" containerName="maas-api"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.314012    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f909dc40-3f8d-4644-b8e0-f8484f347ced" containerName="payload-pre-processing"
Sep 10 12:08:15.314225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.314023    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="35fa68b1-1bef-4594-beed-56f1fe4b54eb" containerName="main"
Sep 10 12:08:15.317217 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.317197    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.319817 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.319798    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-316319-dockercfg-gcwzr\""
Sep 10 12:08:15.320014 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.319996    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-316319\""
Sep 10 12:08:15.331124 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.331102    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"]
Sep 10 12:08:15.358453 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.358425    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"]
Sep 10 12:08:15.358792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.358780    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerName="istio-proxy"
Sep 10 12:08:15.361738 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.361719    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.373024 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.373001    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"]
Sep 10 12:08:15.418518 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.418486    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck9st\" (UniqueName: \"kubernetes.io/projected/c465b92a-9171-4c0a-8022-878e464b332b-kube-api-access-ck9st\") pod \"payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.418693 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.418529    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c465b92a-9171-4c0a-8022-878e464b332b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.519277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.519179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck9st\" (UniqueName: \"kubernetes.io/projected/c465b92a-9171-4c0a-8022-878e464b332b-kube-api-access-ck9st\") pod \"payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.519277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.519226    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c465b92a-9171-4c0a-8022-878e464b332b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.519277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.519269    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa30db5-bde3-4b71-bf0f-596c20eb853c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") " pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.519544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.519316    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czksj\" (UniqueName: \"kubernetes.io/projected/ffa30db5-bde3-4b71-bf0f-596c20eb853c-kube-api-access-czksj\") pod \"payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") " pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.519977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.519955    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c465b92a-9171-4c0a-8022-878e464b332b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.527593 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.527569    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck9st\" (UniqueName: \"kubernetes.io/projected/c465b92a-9171-4c0a-8022-878e464b332b-kube-api-access-ck9st\") pod \"payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.619916 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.619884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa30db5-bde3-4b71-bf0f-596c20eb853c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") " pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.620084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.619941    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czksj\" (UniqueName: \"kubernetes.io/projected/ffa30db5-bde3-4b71-bf0f-596c20eb853c-kube-api-access-czksj\") pod \"payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") " pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.620598 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.620579    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa30db5-bde3-4b71-bf0f-596c20eb853c-plugins-config-volume\") pod \"payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") " pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.627730 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.627706    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:15.628735 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.628715    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czksj\" (UniqueName: \"kubernetes.io/projected/ffa30db5-bde3-4b71-bf0f-596c20eb853c-kube-api-access-czksj\") pod \"payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") " pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.672251 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.672218    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:15.682726 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.682669    2571 generic.go:356] "Generic (PLEG): container finished" podID="7fc15578-77f0-4ed8-932e-6a94d42e1411" containerID="59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59" exitCode=0
Sep 10 12:08:15.682844 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.682728    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" event={"ID":"7fc15578-77f0-4ed8-932e-6a94d42e1411","Type":"ContainerDied","Data":"59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59"}
Sep 10 12:08:15.682844 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.682779    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g" event={"ID":"7fc15578-77f0-4ed8-932e-6a94d42e1411","Type":"ContainerDied","Data":"08d8ac079a31b6ce60ff669022206334acf132b2c0519ee7a11a7e672f612185"}
Sep 10 12:08:15.682844 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.682808    2571 scope.go:117] "RemoveContainer" containerID="59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59"
Sep 10 12:08:15.683679 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.682716    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"
Sep 10 12:08:15.701748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.701696    2571 scope.go:117] "RemoveContainer" containerID="5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"
Sep 10 12:08:15.713648 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.713628    2571 scope.go:117] "RemoveContainer" containerID="59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59"
Sep 10 12:08:15.713954 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:08:15.713931    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59\": container with ID starting with 59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59 not found: ID does not exist" containerID="59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59"
Sep 10 12:08:15.714024 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.713964    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59"} err="failed to get container status \"59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59\": rpc error: code = NotFound desc = could not find container \"59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59\": container with ID starting with 59c48a38201138a8f0a4397ac05875fd1141edc265c1f20153f3eda7be478e59 not found: ID does not exist"
Sep 10 12:08:15.714024 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.713984    2571 scope.go:117] "RemoveContainer" containerID="5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"
Sep 10 12:08:15.714323 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:08:15.714297    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f\": container with ID starting with 5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f not found: ID does not exist" containerID="5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"
Sep 10 12:08:15.714395 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.714330    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f"} err="failed to get container status \"5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f\": rpc error: code = NotFound desc = could not find container \"5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f\": container with ID starting with 5fb19cd3baa83f72d6b2854e35017f788fa732ca1dc9db7018d1d718acab627f not found: ID does not exist"
Sep 10 12:08:15.714968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.714928    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"]
Sep 10 12:08:15.721105 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.721084    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-7608bc-openshift-default-7d4d48b7cd-g7g2g"]
Sep 10 12:08:15.892423 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.892346    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fc15578-77f0-4ed8-932e-6a94d42e1411" path="/var/lib/kubelet/pods/7fc15578-77f0-4ed8-932e-6a94d42e1411/volumes"
Sep 10 12:08:15.901143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.901121    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"]
Sep 10 12:08:15.902339 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:08:15.902310    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc465b92a_9171_4c0a_8022_878e464b332b.slice/crio-8ba2e363e21db84d67789f1ecb37f7030f14646116406cb144b93359ca6ba850 WatchSource:0}: Error finding container 8ba2e363e21db84d67789f1ecb37f7030f14646116406cb144b93359ca6ba850: Status 404 returned error can't find the container with id 8ba2e363e21db84d67789f1ecb37f7030f14646116406cb144b93359ca6ba850
Sep 10 12:08:15.975824 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:15.975800    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"]
Sep 10 12:08:15.976197 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:08:15.976174    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa30db5_bde3_4b71_bf0f_596c20eb853c.slice/crio-a9bb30ac685511108d8bed3ff40d5c2c70cc96b8c866146189488b631bdd8971 WatchSource:0}: Error finding container a9bb30ac685511108d8bed3ff40d5c2c70cc96b8c866146189488b631bdd8971: Status 404 returned error can't find the container with id a9bb30ac685511108d8bed3ff40d5c2c70cc96b8c866146189488b631bdd8971
Sep 10 12:08:16.643430 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.643395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:16.691318 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.691282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj" event={"ID":"c465b92a-9171-4c0a-8022-878e464b332b","Type":"ContainerStarted","Data":"a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323"}
Sep 10 12:08:16.691499 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.691348    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj" event={"ID":"c465b92a-9171-4c0a-8022-878e464b332b","Type":"ContainerStarted","Data":"8ba2e363e21db84d67789f1ecb37f7030f14646116406cb144b93359ca6ba850"}
Sep 10 12:08:16.691499 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.691441    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:16.694067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.694043    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg" event={"ID":"ffa30db5-bde3-4b71-bf0f-596c20eb853c","Type":"ContainerStarted","Data":"cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee"}
Sep 10 12:08:16.694198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.694087    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg" event={"ID":"ffa30db5-bde3-4b71-bf0f-596c20eb853c","Type":"ContainerStarted","Data":"a9bb30ac685511108d8bed3ff40d5c2c70cc96b8c866146189488b631bdd8971"}
Sep 10 12:08:16.694198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.694188    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:16.729912 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.729863    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg" podStartSLOduration=1.729849476 podStartE2EDuration="1.729849476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:08:16.728519593 +0000 UTC m=+1007.331794496" watchObservedRunningTime="2026-09-10 12:08:16.729849476 +0000 UTC m=+1007.333124426"
Sep 10 12:08:16.731260 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:16.731224    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj" podStartSLOduration=1.731212834 podStartE2EDuration="1.731212834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:08:16.710069703 +0000 UTC m=+1007.313344627" watchObservedRunningTime="2026-09-10 12:08:16.731212834 +0000 UTC m=+1007.334487745"
Sep 10 12:08:22.848610 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:22.848569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:22.915844 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:22.915807    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:23.015919 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:23.015877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:23.145032 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:23.144955    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:27.702181 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:27.702127    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:08:27.702626 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:27.702215    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:08:33.638862 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:33.638826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:34.643795 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:34.643757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:38.520020 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:38.519980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:40.461601 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:40.461542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:40.530097 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:40.530066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:40.658544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:40.658504    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:40.856409 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:40.856321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:41.442830 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:41.442786    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:41.542766 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:41.542734    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:41.620803 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:41.620773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:43.267116 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:43.267082    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:45.419015 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:45.418981    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:48.655559 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:48.655521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:49.266407 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:49.266375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:49.371683 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:49.371649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:49.970146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:49.970116    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:56.141607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:56.141576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:56.806826 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:56.806790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:56.896707 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:56.896672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:57.057132 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:57.057037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:57.458120 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:57.458080    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:08:57.575430 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:08:57.575399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:00.889698 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:00.889669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:01.019405 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:01.019368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:02.006004 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.005971    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"]
Sep 10 12:09:02.010080 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.010057    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.012505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.012480    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0f6353c1-serving-cert\""
Sep 10 12:09:02.028353 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.028322    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"]
Sep 10 12:09:02.054644 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.054607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:02.131200 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.131143    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j24d7\" (UniqueName: \"kubernetes.io/projected/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-kube-api-access-j24d7\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.131367 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.131243    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-maas-api-tls\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.131367 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.131302    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-metrics-certs\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.131444 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.131378    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-openshift-service-ca\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.232604 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.232572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-openshift-service-ca\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.232825 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.232631    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j24d7\" (UniqueName: \"kubernetes.io/projected/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-kube-api-access-j24d7\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.232825 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.232663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-maas-api-tls\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.232825 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.232704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-metrics-certs\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.233333 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.233301    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-openshift-service-ca\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.235078 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.235054    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-metrics-certs\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.235201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.235173    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-maas-api-tls\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.240598 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.240572    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j24d7\" (UniqueName: \"kubernetes.io/projected/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-kube-api-access-j24d7\") pod \"maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.321619 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.321532    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:02.417119 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.417086    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:02.516919 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.516866    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:02.596201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.596172    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"]
Sep 10 12:09:02.597777 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:09:02.597750    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6ca5b0_8767_483f_9df8_b0d1cf351fcb.slice/crio-390500bd4087b3b28868be94389391cfd8cae8c7ac35a25a73f6c2a056decc9d WatchSource:0}: Error finding container 390500bd4087b3b28868be94389391cfd8cae8c7ac35a25a73f6c2a056decc9d: Status 404 returned error can't find the container with id 390500bd4087b3b28868be94389391cfd8cae8c7ac35a25a73f6c2a056decc9d
Sep 10 12:09:02.599624 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.599608    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:09:02.822828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.822791    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:02.864574 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.864496    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"]
Sep 10 12:09:02.868986 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.868969    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:02.871543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.871521    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0f6353c1-dockercfg-gclfw\""
Sep 10 12:09:02.871637 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.871586    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0f6353c1\""
Sep 10 12:09:02.877087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.877062    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"]
Sep 10 12:09:02.914040 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.914008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:02.933523 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:02.933492    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs" event={"ID":"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb","Type":"ContainerStarted","Data":"390500bd4087b3b28868be94389391cfd8cae8c7ac35a25a73f6c2a056decc9d"}
Sep 10 12:09:03.041381 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.041341    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe7829a-682c-442e-8949-5b0ced49c8b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.041758 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.041394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdjxl\" (UniqueName: \"kubernetes.io/projected/dfe7829a-682c-442e-8949-5b0ced49c8b5-kube-api-access-cdjxl\") pod \"payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.142063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.141980    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe7829a-682c-442e-8949-5b0ced49c8b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.142063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.142030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdjxl\" (UniqueName: \"kubernetes.io/projected/dfe7829a-682c-442e-8949-5b0ced49c8b5-kube-api-access-cdjxl\") pod \"payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.142762 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.142740    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe7829a-682c-442e-8949-5b0ced49c8b5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.151215 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.151188    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdjxl\" (UniqueName: \"kubernetes.io/projected/dfe7829a-682c-442e-8949-5b0ced49c8b5-kube-api-access-cdjxl\") pod \"payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.181116 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.181094    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.460712 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.460686    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"]
Sep 10 12:09:03.461900 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:09:03.461867    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe7829a_682c_442e_8949_5b0ced49c8b5.slice/crio-a1f6ec1b42b3dafaf471be47f250144787d3ed40fa12656177b17f491ed63692 WatchSource:0}: Error finding container a1f6ec1b42b3dafaf471be47f250144787d3ed40fa12656177b17f491ed63692: Status 404 returned error can't find the container with id a1f6ec1b42b3dafaf471be47f250144787d3ed40fa12656177b17f491ed63692
Sep 10 12:09:03.940037 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.939988    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs" event={"ID":"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb","Type":"ContainerStarted","Data":"5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c"}
Sep 10 12:09:03.940230 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.940069    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:03.941606 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.941581    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d" event={"ID":"dfe7829a-682c-442e-8949-5b0ced49c8b5","Type":"ContainerStarted","Data":"81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1"}
Sep 10 12:09:03.941719 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.941630    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d" event={"ID":"dfe7829a-682c-442e-8949-5b0ced49c8b5","Type":"ContainerStarted","Data":"a1f6ec1b42b3dafaf471be47f250144787d3ed40fa12656177b17f491ed63692"}
Sep 10 12:09:03.941719 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.941655    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:03.961217 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.961169    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs" podStartSLOduration=2.031667987 podStartE2EDuration="2.961138553s" totalImagesPullingTime="929.470566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:03.959328432 +0000 UTC m=+1054.562603356" watchObservedRunningTime="2026-09-10 12:09:03.961138553 +0000 UTC m=+1054.564413467"
Sep 10 12:09:03.980827 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:03.980785    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d" podStartSLOduration=1.980772003 podStartE2EDuration="1.980772003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:03.979429708 +0000 UTC m=+1054.582704621" watchObservedRunningTime="2026-09-10 12:09:03.980772003 +0000 UTC m=+1054.584046915"
Sep 10 12:09:06.508286 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:06.508255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:07.114456 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.114421    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:07.143407 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.143376    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"]
Sep 10 12:09:07.143654 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.143630    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg" podUID="ffa30db5-bde3-4b71-bf0f-596c20eb853c" containerName="payload-processing" containerID="cri-o://cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee" gracePeriod=30
Sep 10 12:09:07.151121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.151098    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"]
Sep 10 12:09:07.151516 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.151487    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj" podUID="c465b92a-9171-4c0a-8022-878e464b332b" containerName="payload-pre-processing" containerID="cri-o://a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323" gracePeriod=30
Sep 10 12:09:07.218609 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.218570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:07.317883 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.317653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:07.416524 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.416432    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:07.426338 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.426313    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:09:07.429850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.429830    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:09:07.587733 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.587701    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck9st\" (UniqueName: \"kubernetes.io/projected/c465b92a-9171-4c0a-8022-878e464b332b-kube-api-access-ck9st\") pod \"c465b92a-9171-4c0a-8022-878e464b332b\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") "
Sep 10 12:09:07.588208 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.587749    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czksj\" (UniqueName: \"kubernetes.io/projected/ffa30db5-bde3-4b71-bf0f-596c20eb853c-kube-api-access-czksj\") pod \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") "
Sep 10 12:09:07.588208 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.587884    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c465b92a-9171-4c0a-8022-878e464b332b-plugins-config-volume\") pod \"c465b92a-9171-4c0a-8022-878e464b332b\" (UID: \"c465b92a-9171-4c0a-8022-878e464b332b\") "
Sep 10 12:09:07.588208 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.587953    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa30db5-bde3-4b71-bf0f-596c20eb853c-plugins-config-volume\") pod \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\" (UID: \"ffa30db5-bde3-4b71-bf0f-596c20eb853c\") "
Sep 10 12:09:07.588348 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.588266    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c465b92a-9171-4c0a-8022-878e464b332b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c465b92a-9171-4c0a-8022-878e464b332b" (UID: "c465b92a-9171-4c0a-8022-878e464b332b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:07.588390 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.588346    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffa30db5-bde3-4b71-bf0f-596c20eb853c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ffa30db5-bde3-4b71-bf0f-596c20eb853c" (UID: "ffa30db5-bde3-4b71-bf0f-596c20eb853c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:07.589932 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.589909    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa30db5-bde3-4b71-bf0f-596c20eb853c-kube-api-access-czksj" (OuterVolumeSpecName: "kube-api-access-czksj") pod "ffa30db5-bde3-4b71-bf0f-596c20eb853c" (UID: "ffa30db5-bde3-4b71-bf0f-596c20eb853c"). InnerVolumeSpecName "kube-api-access-czksj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:07.590189 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.590172    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c465b92a-9171-4c0a-8022-878e464b332b-kube-api-access-ck9st" (OuterVolumeSpecName: "kube-api-access-ck9st") pod "c465b92a-9171-4c0a-8022-878e464b332b" (UID: "c465b92a-9171-4c0a-8022-878e464b332b"). InnerVolumeSpecName "kube-api-access-ck9st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:07.688787 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.688741    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c465b92a-9171-4c0a-8022-878e464b332b-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:07.688787 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.688788    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffa30db5-bde3-4b71-bf0f-596c20eb853c-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:07.688993 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.688798    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck9st\" (UniqueName: \"kubernetes.io/projected/c465b92a-9171-4c0a-8022-878e464b332b-kube-api-access-ck9st\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:07.688993 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.688806    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czksj\" (UniqueName: \"kubernetes.io/projected/ffa30db5-bde3-4b71-bf0f-596c20eb853c-kube-api-access-czksj\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:07.689401 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.689375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:07.963440 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.963408    2571 generic.go:356] "Generic (PLEG): container finished" podID="ffa30db5-bde3-4b71-bf0f-596c20eb853c" containerID="cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee" exitCode=0
Sep 10 12:09:07.963592 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.963464    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"
Sep 10 12:09:07.963592 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.963492    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg" event={"ID":"ffa30db5-bde3-4b71-bf0f-596c20eb853c","Type":"ContainerDied","Data":"cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee"}
Sep 10 12:09:07.963592 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.963551    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg" event={"ID":"ffa30db5-bde3-4b71-bf0f-596c20eb853c","Type":"ContainerDied","Data":"a9bb30ac685511108d8bed3ff40d5c2c70cc96b8c866146189488b631bdd8971"}
Sep 10 12:09:07.963592 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.963582    2571 scope.go:117] "RemoveContainer" containerID="cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee"
Sep 10 12:09:07.964862 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.964842    2571 generic.go:356] "Generic (PLEG): container finished" podID="c465b92a-9171-4c0a-8022-878e464b332b" containerID="a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323" exitCode=0
Sep 10 12:09:07.964945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.964905    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"
Sep 10 12:09:07.964945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.964919    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj" event={"ID":"c465b92a-9171-4c0a-8022-878e464b332b","Type":"ContainerDied","Data":"a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323"}
Sep 10 12:09:07.965048 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.964956    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj" event={"ID":"c465b92a-9171-4c0a-8022-878e464b332b","Type":"ContainerDied","Data":"8ba2e363e21db84d67789f1ecb37f7030f14646116406cb144b93359ca6ba850"}
Sep 10 12:09:07.972937 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.972916    2571 scope.go:117] "RemoveContainer" containerID="cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee"
Sep 10 12:09:07.973212 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:07.973194    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee\": container with ID starting with cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee not found: ID does not exist" containerID="cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee"
Sep 10 12:09:07.973279 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.973218    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee"} err="failed to get container status \"cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee\": rpc error: code = NotFound desc = could not find container \"cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee\": container with ID starting with cf3b3ebaf99d04b16cbd9b7d91394af1d983ca9d152e54668d16caf74cb85bee not found: ID does not exist"
Sep 10 12:09:07.973279 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.973232    2571 scope.go:117] "RemoveContainer" containerID="a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323"
Sep 10 12:09:07.980813 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.980797    2571 scope.go:117] "RemoveContainer" containerID="a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323"
Sep 10 12:09:07.981072 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:07.981053    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323\": container with ID starting with a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323 not found: ID does not exist" containerID="a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323"
Sep 10 12:09:07.981115 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.981079    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323"} err="failed to get container status \"a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323\": rpc error: code = NotFound desc = could not find container \"a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323\": container with ID starting with a0c89d634f2c53231874c4ffbf2e80e9f0937c2d25d8c35230fb33bfd1ba4323 not found: ID does not exist"
Sep 10 12:09:07.985336 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.985307    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"]
Sep 10 12:09:07.989463 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:07.989440    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-316319-6954bc45b9-zcwcg"]
Sep 10 12:09:08.006832 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:08.006785    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"]
Sep 10 12:09:08.008802 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:08.008782    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-316319-557d8b7d98-qbrfj"]
Sep 10 12:09:09.892096 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:09.892064    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c465b92a-9171-4c0a-8022-878e464b332b" path="/var/lib/kubelet/pods/c465b92a-9171-4c0a-8022-878e464b332b/volumes"
Sep 10 12:09:09.892496 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:09.892463    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffa30db5-bde3-4b71-bf0f-596c20eb853c" path="/var/lib/kubelet/pods/ffa30db5-bde3-4b71-bf0f-596c20eb853c/volumes"
Sep 10 12:09:09.953407 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:09.953383    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:12.762137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:12.762106    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:14.949919 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:14.949890    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:15.682475 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:15.682445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:16.841671 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:16.841619    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:17.085410 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:17.085334    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:19.073175 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:19.073131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:19.334587 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:19.334492    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:19.777676 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:19.777637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:20.085079 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:20.084999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:20.237109 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:20.237075    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:21.454355 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:21.454319    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:21.565053 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:21.565019    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:21.696199 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:21.696145    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:23.101095 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:23.101008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:25.833137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:25.833100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:26.535480 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:26.535440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:26.626451 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:26.626399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:26.962363 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:26.962307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:27.071092 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.071049    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:27.667772 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.667739    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"]
Sep 10 12:09:27.667997 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.667942    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l" podUID="0a21f5be-1500-4e8f-b073-4dfdb617f4f4" containerName="maas-api" containerID="cri-o://ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6" gracePeriod=30
Sep 10 12:09:27.701884 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.701860    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:27.802503 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.802473    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:27.905200 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.905173    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:27.922651 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.922595    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:09:27.977690 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.977657    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-maas-api-tls\") pod \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") "
Sep 10 12:09:27.978146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.977719    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-openshift-service-ca\") pod \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") "
Sep 10 12:09:27.978146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.977747    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-metrics-certs\") pod \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") "
Sep 10 12:09:27.978146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.977914    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgwjc\" (UniqueName: \"kubernetes.io/projected/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-kube-api-access-zgwjc\") pod \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\" (UID: \"0a21f5be-1500-4e8f-b073-4dfdb617f4f4\") "
Sep 10 12:09:27.978146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.978108    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a21f5be-1500-4e8f-b073-4dfdb617f4f4" (UID: "0a21f5be-1500-4e8f-b073-4dfdb617f4f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:27.978475 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.978258    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:27.979983 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.979955    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a21f5be-1500-4e8f-b073-4dfdb617f4f4" (UID: "0a21f5be-1500-4e8f-b073-4dfdb617f4f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:09:27.980093 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.980062    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0a21f5be-1500-4e8f-b073-4dfdb617f4f4" (UID: "0a21f5be-1500-4e8f-b073-4dfdb617f4f4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:09:27.980093 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:27.980079    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-kube-api-access-zgwjc" (OuterVolumeSpecName: "kube-api-access-zgwjc") pod "0a21f5be-1500-4e8f-b073-4dfdb617f4f4" (UID: "0a21f5be-1500-4e8f-b073-4dfdb617f4f4"). InnerVolumeSpecName "kube-api-access-zgwjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:28.011663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.011636    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:28.087942 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.084371    2571 generic.go:356] "Generic (PLEG): container finished" podID="0a21f5be-1500-4e8f-b073-4dfdb617f4f4" containerID="ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6" exitCode=0
Sep 10 12:09:28.087942 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.084536    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l" event={"ID":"0a21f5be-1500-4e8f-b073-4dfdb617f4f4","Type":"ContainerDied","Data":"ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6"}
Sep 10 12:09:28.087942 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.084585    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l" event={"ID":"0a21f5be-1500-4e8f-b073-4dfdb617f4f4","Type":"ContainerDied","Data":"3bbf8d4d4961f2f51c016049f6b8520e015ffdbd5f77dfb31ec9e662c8390dd7"}
Sep 10 12:09:28.087942 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.084613    2571 scope.go:117] "RemoveContainer" containerID="ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6"
Sep 10 12:09:28.087942 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.084778    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"
Sep 10 12:09:28.089962 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.089931    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:28.090120 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.089964    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:28.090120 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.089983    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgwjc\" (UniqueName: \"kubernetes.io/projected/0a21f5be-1500-4e8f-b073-4dfdb617f4f4-kube-api-access-zgwjc\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:28.099043 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.099019    2571 scope.go:117] "RemoveContainer" containerID="ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6"
Sep 10 12:09:28.099375 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:28.099350    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6\": container with ID starting with ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6 not found: ID does not exist" containerID="ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6"
Sep 10 12:09:28.099437 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.099386    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6"} err="failed to get container status \"ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6\": rpc error: code = NotFound desc = could not find container \"ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6\": container with ID starting with ee5ced8d353be353ec257e87f5f17568a2baf3247038bfcd2a129df6ba1881b6 not found: ID does not exist"
Sep 10 12:09:28.111930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.111898    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"]
Sep 10 12:09:28.117868 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.117828    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-89d91a-674d8595dd-dgt8l"]
Sep 10 12:09:28.269299 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:28.269264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:29.892669 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:29.892639    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a21f5be-1500-4e8f-b073-4dfdb617f4f4" path="/var/lib/kubelet/pods/0a21f5be-1500-4e8f-b073-4dfdb617f4f4/volumes"
Sep 10 12:09:33.189245 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:33.189197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:33.288801 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:33.288769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:33.805858 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:33.805815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:35.139338 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:35.139302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:35.237873 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:35.237840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:35.379479 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:35.379445    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"]
Sep 10 12:09:35.379713 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:35.379688    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy" containerID="cri-o://c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d" gracePeriod=30
Sep 10 12:09:35.407402 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:35.407316    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:36.063397 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:36.063361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:40.627899 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.627871    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:09:40.803760 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803728    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-token\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.803760 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803765    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-data\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803788    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77293c6d-96be-4e1d-ad18-b78993e11fdb-istiod-ca-cert\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803830    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-envoy\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803853    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-certs\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803874    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vf25\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-kube-api-access-6vf25\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803916    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-podinfo\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803961    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-credential-socket\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.803983    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-socket\") pod \"77293c6d-96be-4e1d-ad18-b78993e11fdb\" (UID: \"77293c6d-96be-4e1d-ad18-b78993e11fdb\") "
Sep 10 12:09:40.804448 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.804033    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-data" (OuterVolumeSpecName: "istio-data") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:09:40.804448 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.804309    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:09:40.804448 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.804316    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-data\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.804618 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.804484    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:09:40.804618 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.804588    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:09:40.804724 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.804663    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77293c6d-96be-4e1d-ad18-b78993e11fdb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:40.806020 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.805991    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-kube-api-access-6vf25" (OuterVolumeSpecName: "kube-api-access-6vf25") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "kube-api-access-6vf25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:40.806213 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.806095    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:09:40.806413 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.806393    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 12:09:40.806499 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.806416    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-token" (OuterVolumeSpecName: "istio-token") pod "77293c6d-96be-4e1d-ad18-b78993e11fdb" (UID: "77293c6d-96be-4e1d-ad18-b78993e11fdb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:40.892238 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.892203    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:40.905446 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905418    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-envoy\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905446 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905442    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905452    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vf25\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-kube-api-access-6vf25\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905463    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-podinfo\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905471    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-credential-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905478    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77293c6d-96be-4e1d-ad18-b78993e11fdb-workload-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905489    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77293c6d-96be-4e1d-ad18-b78993e11fdb-istio-token\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:40.905607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:40.905502    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77293c6d-96be-4e1d-ad18-b78993e11fdb-istiod-ca-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:41.152276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.152186    2571 generic.go:356] "Generic (PLEG): container finished" podID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerID="c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d" exitCode=0
Sep 10 12:09:41.152276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.152233    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"
Sep 10 12:09:41.152455 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.152234    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" event={"ID":"77293c6d-96be-4e1d-ad18-b78993e11fdb","Type":"ContainerDied","Data":"c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d"}
Sep 10 12:09:41.152455 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.152369    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7" event={"ID":"77293c6d-96be-4e1d-ad18-b78993e11fdb","Type":"ContainerDied","Data":"06b9a8daea5779831147c5aa21023d9b884d1a5d3653c6da9290afb3f67d7f97"}
Sep 10 12:09:41.152455 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.152409    2571 scope.go:117] "RemoveContainer" containerID="c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d"
Sep 10 12:09:41.161801 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.161784    2571 scope.go:117] "RemoveContainer" containerID="8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"
Sep 10 12:09:41.169702 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.169683    2571 scope.go:117] "RemoveContainer" containerID="c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d"
Sep 10 12:09:41.169954 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:41.169936    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d\": container with ID starting with c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d not found: ID does not exist" containerID="c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d"
Sep 10 12:09:41.169997 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.169964    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d"} err="failed to get container status \"c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d\": rpc error: code = NotFound desc = could not find container \"c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d\": container with ID starting with c6d197ee7ef057bf7137a0ed93e9a62295f3a588864bb0e5cf85439cc27d418d not found: ID does not exist"
Sep 10 12:09:41.169997 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.169981    2571 scope.go:117] "RemoveContainer" containerID="8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"
Sep 10 12:09:41.170270 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:41.170253    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702\": container with ID starting with 8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702 not found: ID does not exist" containerID="8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"
Sep 10 12:09:41.170314 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.170277    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702"} err="failed to get container status \"8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702\": rpc error: code = NotFound desc = could not find container \"8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702\": container with ID starting with 8097b81274610fe349f5f469ce6f12bf002b740e5e6ca9f34c1c77159a032702 not found: ID does not exist"
Sep 10 12:09:41.175064 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.175044    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"]
Sep 10 12:09:41.182003 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.181978    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-89d91a-openshift-default-7ff5b45474-pjhw7"]
Sep 10 12:09:41.604142 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.604110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:41.707092 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.707063    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:41.894704 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:41.894628    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" path="/var/lib/kubelet/pods/77293c6d-96be-4e1d-ad18-b78993e11fdb/volumes"
Sep 10 12:09:43.683322 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:43.683285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:44.361027 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.360999    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"]
Sep 10 12:09:44.361289 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.361254    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs" podUID="1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" containerName="maas-api" containerID="cri-o://5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c" gracePeriod=30
Sep 10 12:09:44.393368 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.393336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:44.426215 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.426170    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"]
Sep 10 12:09:44.426425 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.426395    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d" podUID="dfe7829a-682c-442e-8949-5b0ced49c8b5" containerName="payload-pre-processing" containerID="cri-o://81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1" gracePeriod=30
Sep 10 12:09:44.521211 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.521177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:44.602203 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.602170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:44.663904 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.663879    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:44.670420 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.670393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:44.704426 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.704404    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:44.739380 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739355    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-metrics-certs\") pod \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") "
Sep 10 12:09:44.739552 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739394    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-openshift-service-ca\") pod \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") "
Sep 10 12:09:44.739552 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739435    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j24d7\" (UniqueName: \"kubernetes.io/projected/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-kube-api-access-j24d7\") pod \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") "
Sep 10 12:09:44.739552 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739501    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe7829a-682c-442e-8949-5b0ced49c8b5-plugins-config-volume\") pod \"dfe7829a-682c-442e-8949-5b0ced49c8b5\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") "
Sep 10 12:09:44.739552 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739549    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdjxl\" (UniqueName: \"kubernetes.io/projected/dfe7829a-682c-442e-8949-5b0ced49c8b5-kube-api-access-cdjxl\") pod \"dfe7829a-682c-442e-8949-5b0ced49c8b5\" (UID: \"dfe7829a-682c-442e-8949-5b0ced49c8b5\") "
Sep 10 12:09:44.739758 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739580    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-maas-api-tls\") pod \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\" (UID: \"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb\") "
Sep 10 12:09:44.739863 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739840    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" (UID: "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:44.739939 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.739922    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfe7829a-682c-442e-8949-5b0ced49c8b5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dfe7829a-682c-442e-8949-5b0ced49c8b5" (UID: "dfe7829a-682c-442e-8949-5b0ced49c8b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:44.741618 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.741590    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-kube-api-access-j24d7" (OuterVolumeSpecName: "kube-api-access-j24d7") pod "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" (UID: "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb"). InnerVolumeSpecName "kube-api-access-j24d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:44.741722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.741643    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" (UID: "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:09:44.741722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.741698    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfe7829a-682c-442e-8949-5b0ced49c8b5-kube-api-access-cdjxl" (OuterVolumeSpecName: "kube-api-access-cdjxl") pod "dfe7829a-682c-442e-8949-5b0ced49c8b5" (UID: "dfe7829a-682c-442e-8949-5b0ced49c8b5"). InnerVolumeSpecName "kube-api-access-cdjxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:44.741919 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.741892    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" (UID: "1d6ca5b0-8767-483f-9df8-b0d1cf351fcb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:09:44.840610 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.840580    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfe7829a-682c-442e-8949-5b0ced49c8b5-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:44.840610 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.840605    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdjxl\" (UniqueName: \"kubernetes.io/projected/dfe7829a-682c-442e-8949-5b0ced49c8b5-kube-api-access-cdjxl\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:44.840792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.840619    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:44.840792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.840631    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:44.840792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.840642    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:44.840792 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.840655    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j24d7\" (UniqueName: \"kubernetes.io/projected/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb-kube-api-access-j24d7\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:44.973056 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:44.973025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:45.174329 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.174298    2571 generic.go:356] "Generic (PLEG): container finished" podID="dfe7829a-682c-442e-8949-5b0ced49c8b5" containerID="81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1" exitCode=0
Sep 10 12:09:45.174541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.174341    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"
Sep 10 12:09:45.174541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.174381    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d" event={"ID":"dfe7829a-682c-442e-8949-5b0ced49c8b5","Type":"ContainerDied","Data":"81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1"}
Sep 10 12:09:45.174541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.174438    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d" event={"ID":"dfe7829a-682c-442e-8949-5b0ced49c8b5","Type":"ContainerDied","Data":"a1f6ec1b42b3dafaf471be47f250144787d3ed40fa12656177b17f491ed63692"}
Sep 10 12:09:45.174541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.174467    2571 scope.go:117] "RemoveContainer" containerID="81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1"
Sep 10 12:09:45.175825 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.175797    2571 generic.go:356] "Generic (PLEG): container finished" podID="1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" containerID="5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c" exitCode=0
Sep 10 12:09:45.175964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.175837    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"
Sep 10 12:09:45.175964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.175876    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs" event={"ID":"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb","Type":"ContainerDied","Data":"5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c"}
Sep 10 12:09:45.175964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.175914    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs" event={"ID":"1d6ca5b0-8767-483f-9df8-b0d1cf351fcb","Type":"ContainerDied","Data":"390500bd4087b3b28868be94389391cfd8cae8c7ac35a25a73f6c2a056decc9d"}
Sep 10 12:09:45.184725 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.184706    2571 scope.go:117] "RemoveContainer" containerID="81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1"
Sep 10 12:09:45.184986 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:45.184967    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1\": container with ID starting with 81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1 not found: ID does not exist" containerID="81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1"
Sep 10 12:09:45.185067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.184998    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1"} err="failed to get container status \"81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1\": rpc error: code = NotFound desc = could not find container \"81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1\": container with ID starting with 81605ffb07bc4f083b410bc85dae367f26edf0acc67d1503b6abb3fdf48f6cd1 not found: ID does not exist"
Sep 10 12:09:45.185067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.185023    2571 scope.go:117] "RemoveContainer" containerID="5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c"
Sep 10 12:09:45.192889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.192864    2571 scope.go:117] "RemoveContainer" containerID="5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c"
Sep 10 12:09:45.193126 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:45.193106    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c\": container with ID starting with 5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c not found: ID does not exist" containerID="5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c"
Sep 10 12:09:45.193207 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.193133    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c"} err="failed to get container status \"5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c\": rpc error: code = NotFound desc = could not find container \"5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c\": container with ID starting with 5acd0e228545960e88f7a8f348dd429b808f4821ad7fe760c81d0c0cbedf459c not found: ID does not exist"
Sep 10 12:09:45.196873 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.196851    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"]
Sep 10 12:09:45.201338 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.201313    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0f6353c1-756f9cccf4-j4p5d"]
Sep 10 12:09:45.212714 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.212689    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"]
Sep 10 12:09:45.216718 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.216687    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0f6353c1-76fd68b957-lfncs"]
Sep 10 12:09:45.893084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.893048    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" path="/var/lib/kubelet/pods/1d6ca5b0-8767-483f-9df8-b0d1cf351fcb/volumes"
Sep 10 12:09:45.893465 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.893438    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfe7829a-682c-442e-8949-5b0ced49c8b5" path="/var/lib/kubelet/pods/dfe7829a-682c-442e-8949-5b0ced49c8b5/volumes"
Sep 10 12:09:45.935571 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:45.935546    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:46.640321 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.640287    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"]
Sep 10 12:09:46.640553 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.640507    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj" podUID="ae7338fa-25a3-40d2-a297-e90e3a4bb17f" containerName="maas-api" containerID="cri-o://131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384" gracePeriod=30
Sep 10 12:09:46.683584 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.683553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:46.738021 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.737992    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"]
Sep 10 12:09:46.738232 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.738211    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk" podUID="74cad910-d8b0-4281-87c4-36c91ae80d75" containerName="payload-pre-processing" containerID="cri-o://d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b" gracePeriod=30
Sep 10 12:09:46.893204 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.893129    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:09:46.955861 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.955830    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-openshift-service-ca\") pod \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") "
Sep 10 12:09:46.955861 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.955873    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-maas-api-tls\") pod \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") "
Sep 10 12:09:46.956114 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.955923    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9th87\" (UniqueName: \"kubernetes.io/projected/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-kube-api-access-9th87\") pod \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") "
Sep 10 12:09:46.956248 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.956222    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae7338fa-25a3-40d2-a297-e90e3a4bb17f" (UID: "ae7338fa-25a3-40d2-a297-e90e3a4bb17f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:46.958311 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.958278    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae7338fa-25a3-40d2-a297-e90e3a4bb17f" (UID: "ae7338fa-25a3-40d2-a297-e90e3a4bb17f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:09:46.958456 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.958411    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-kube-api-access-9th87" (OuterVolumeSpecName: "kube-api-access-9th87") pod "ae7338fa-25a3-40d2-a297-e90e3a4bb17f" (UID: "ae7338fa-25a3-40d2-a297-e90e3a4bb17f"). InnerVolumeSpecName "kube-api-access-9th87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:46.992606 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:46.992584    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:09:47.057297 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057257    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-metrics-certs\") pod \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\" (UID: \"ae7338fa-25a3-40d2-a297-e90e3a4bb17f\") "
Sep 10 12:09:47.057452 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057306    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdd88\" (UniqueName: \"kubernetes.io/projected/74cad910-d8b0-4281-87c4-36c91ae80d75-kube-api-access-kdd88\") pod \"74cad910-d8b0-4281-87c4-36c91ae80d75\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") "
Sep 10 12:09:47.057452 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057365    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cad910-d8b0-4281-87c4-36c91ae80d75-plugins-config-volume\") pod \"74cad910-d8b0-4281-87c4-36c91ae80d75\" (UID: \"74cad910-d8b0-4281-87c4-36c91ae80d75\") "
Sep 10 12:09:47.057539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057489    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:47.057539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057500    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:47.057539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057508    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9th87\" (UniqueName: \"kubernetes.io/projected/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-kube-api-access-9th87\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:47.057770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.057746    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74cad910-d8b0-4281-87c4-36c91ae80d75-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74cad910-d8b0-4281-87c4-36c91ae80d75" (UID: "74cad910-d8b0-4281-87c4-36c91ae80d75"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:09:47.059497 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.059469    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae7338fa-25a3-40d2-a297-e90e3a4bb17f" (UID: "ae7338fa-25a3-40d2-a297-e90e3a4bb17f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:09:47.059497 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.059473    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74cad910-d8b0-4281-87c4-36c91ae80d75-kube-api-access-kdd88" (OuterVolumeSpecName: "kube-api-access-kdd88") pod "74cad910-d8b0-4281-87c4-36c91ae80d75" (UID: "74cad910-d8b0-4281-87c4-36c91ae80d75"). InnerVolumeSpecName "kube-api-access-kdd88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:09:47.059666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.059533    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:47.158623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.158536    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74cad910-d8b0-4281-87c4-36c91ae80d75-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:47.158623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.158567    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae7338fa-25a3-40d2-a297-e90e3a4bb17f-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:47.158623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.158581    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdd88\" (UniqueName: \"kubernetes.io/projected/74cad910-d8b0-4281-87c4-36c91ae80d75-kube-api-access-kdd88\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:09:47.188528 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.188492    2571 generic.go:356] "Generic (PLEG): container finished" podID="ae7338fa-25a3-40d2-a297-e90e3a4bb17f" containerID="131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384" exitCode=0
Sep 10 12:09:47.188689 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.188543    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"
Sep 10 12:09:47.188689 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.188577    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj" event={"ID":"ae7338fa-25a3-40d2-a297-e90e3a4bb17f","Type":"ContainerDied","Data":"131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384"}
Sep 10 12:09:47.188689 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.188632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj" event={"ID":"ae7338fa-25a3-40d2-a297-e90e3a4bb17f","Type":"ContainerDied","Data":"25fb3f7acd38d49ef9e93e47dab34f5829e27b7435e43b3db1113c7c82b912fc"}
Sep 10 12:09:47.188689 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.188658    2571 scope.go:117] "RemoveContainer" containerID="131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384"
Sep 10 12:09:47.189928 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.189909    2571 generic.go:356] "Generic (PLEG): container finished" podID="74cad910-d8b0-4281-87c4-36c91ae80d75" containerID="d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b" exitCode=0
Sep 10 12:09:47.190036 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.189953    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"
Sep 10 12:09:47.190083 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.190040    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk" event={"ID":"74cad910-d8b0-4281-87c4-36c91ae80d75","Type":"ContainerDied","Data":"d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b"}
Sep 10 12:09:47.190118 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.190084    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk" event={"ID":"74cad910-d8b0-4281-87c4-36c91ae80d75","Type":"ContainerDied","Data":"f282a59912160d6a2185728ef62c5334443b8edaf73922dfab863cbd4d34e814"}
Sep 10 12:09:47.197246 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.197141    2571 scope.go:117] "RemoveContainer" containerID="131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384"
Sep 10 12:09:47.197547 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:47.197523    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384\": container with ID starting with 131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384 not found: ID does not exist" containerID="131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384"
Sep 10 12:09:47.197642 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.197550    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384"} err="failed to get container status \"131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384\": rpc error: code = NotFound desc = could not find container \"131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384\": container with ID starting with 131c459a00ff17447b70b318892624c6e30518e601fff0afb94f3d73e2635384 not found: ID does not exist"
Sep 10 12:09:47.197642 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.197566    2571 scope.go:117] "RemoveContainer" containerID="d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b"
Sep 10 12:09:47.205497 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.205479    2571 scope.go:117] "RemoveContainer" containerID="d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b"
Sep 10 12:09:47.205743 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:09:47.205728    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b\": container with ID starting with d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b not found: ID does not exist" containerID="d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b"
Sep 10 12:09:47.205794 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.205748    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b"} err="failed to get container status \"d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b\": rpc error: code = NotFound desc = could not find container \"d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b\": container with ID starting with d75936464dc4365f84781af4d5291257108f9ffe94fd957546fcc0f3ca52b15b not found: ID does not exist"
Sep 10 12:09:47.227122 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.227100    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"]
Sep 10 12:09:47.237977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.237958    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-425dc7-5978f7cc48-km8tj"]
Sep 10 12:09:47.249948 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.249925    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"]
Sep 10 12:09:47.257645 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.257620    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-425dc7-66f5657df4-xbpqk"]
Sep 10 12:09:47.289652 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.289628    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:47.692574 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.692545    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:47.891602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.891569    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74cad910-d8b0-4281-87c4-36c91ae80d75" path="/var/lib/kubelet/pods/74cad910-d8b0-4281-87c4-36c91ae80d75/volumes"
Sep 10 12:09:47.891952 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:47.891940    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae7338fa-25a3-40d2-a297-e90e3a4bb17f" path="/var/lib/kubelet/pods/ae7338fa-25a3-40d2-a297-e90e3a4bb17f/volumes"
Sep 10 12:09:52.545901 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:52.545865    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:52.678633 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:52.678593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:52.901988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:52.901907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:53.346283 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:53.346251    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:53.476091 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:53.476061    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:53.687765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:53.687731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:55.812929 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:55.812898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:55.973511 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:55.973481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:56.148258 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:56.148170    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:56.259891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:56.259859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:56.360955 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:56.360926    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:56.792963 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:56.792932    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:57.126223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126138    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"]
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126440    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a21f5be-1500-4e8f-b073-4dfdb617f4f4" containerName="maas-api"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126452    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a21f5be-1500-4e8f-b073-4dfdb617f4f4" containerName="maas-api"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126464    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfe7829a-682c-442e-8949-5b0ced49c8b5" containerName="payload-pre-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126470    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfe7829a-682c-442e-8949-5b0ced49c8b5" containerName="payload-pre-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126479    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74cad910-d8b0-4281-87c4-36c91ae80d75" containerName="payload-pre-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126485    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="74cad910-d8b0-4281-87c4-36c91ae80d75" containerName="payload-pre-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126490    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126496    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126501    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" containerName="maas-api"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126506    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" containerName="maas-api"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126515    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae7338fa-25a3-40d2-a297-e90e3a4bb17f" containerName="maas-api"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126520    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7338fa-25a3-40d2-a297-e90e3a4bb17f" containerName="maas-api"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126529    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c465b92a-9171-4c0a-8022-878e464b332b" containerName="payload-pre-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126534    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c465b92a-9171-4c0a-8022-878e464b332b" containerName="payload-pre-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126542    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126547    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126552    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa30db5-bde3-4b71-bf0f-596c20eb853c" containerName="payload-processing"
Sep 10 12:09:57.126554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126558    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa30db5-bde3-4b71-bf0f-596c20eb853c" containerName="payload-processing"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126608    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffa30db5-bde3-4b71-bf0f-596c20eb853c" containerName="payload-processing"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126617    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae7338fa-25a3-40d2-a297-e90e3a4bb17f" containerName="maas-api"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126622    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="74cad910-d8b0-4281-87c4-36c91ae80d75" containerName="payload-pre-processing"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126627    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6ca5b0-8767-483f-9df8-b0d1cf351fcb" containerName="maas-api"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126633    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfe7829a-682c-442e-8949-5b0ced49c8b5" containerName="payload-pre-processing"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126639    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126643    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a21f5be-1500-4e8f-b073-4dfdb617f4f4" containerName="maas-api"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126649    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="77293c6d-96be-4e1d-ad18-b78993e11fdb" containerName="istio-proxy"
Sep 10 12:09:57.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.126654    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c465b92a-9171-4c0a-8022-878e464b332b" containerName="payload-pre-processing"
Sep 10 12:09:57.130899 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.130879    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.135218 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.135197    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-c07be0-openshift-default-dockercfg-gpqj7\""
Sep 10 12:09:57.154881 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.154860    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"]
Sep 10 12:09:57.241328 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-credential-socket\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241328 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241332    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-data\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241352    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istiod-ca-cert\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241444    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv25l\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-kube-api-access-bv25l\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241499    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-podinfo\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241631 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-socket\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241631 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-certs\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241631 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241611    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-token\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.241728 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.241637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-envoy\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.253728 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.253702    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:57.342115 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342080    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-socket\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342115 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-certs\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342372 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-token\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342372 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342226    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-envoy\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342372 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342269    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-credential-socket\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342372 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342290    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-data\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342428    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istiod-ca-cert\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342477    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv25l\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-kube-api-access-bv25l\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342523    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-socket\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342540    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-podinfo\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342578 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342560    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-certs\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342819 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342603    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-credential-socket\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.342819 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342626    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-data\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.343007 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.342989    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istiod-ca-cert\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.344463 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.344444    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-envoy\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.344810 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.344793    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-podinfo\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.352995 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.352974    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-token\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.353424 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.353406    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv25l\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-kube-api-access-bv25l\") pod \"e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") " pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.440413 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.440386    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:57.671214 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.671182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:57.723275 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.723252    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"]
Sep 10 12:09:57.724753 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:09:57.724725    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4e3672_c7f4_49d4_aa51_c2ed03041a9a.slice/crio-e78e3b6ed97011daded1402f544b3c05502e946a4a2b3946ab5de321d80dbdb1 WatchSource:0}: Error finding container e78e3b6ed97011daded1402f544b3c05502e946a4a2b3946ab5de321d80dbdb1: Status 404 returned error can't find the container with id e78e3b6ed97011daded1402f544b3c05502e946a4a2b3946ab5de321d80dbdb1
Sep 10 12:09:57.726833 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.726805    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:09:57.726906 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.726866    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:09:57.726906 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:57.726896    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:09:58.248905 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:58.248860    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" event={"ID":"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a","Type":"ContainerStarted","Data":"9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"}
Sep 10 12:09:58.249369 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:58.248934    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" event={"ID":"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a","Type":"ContainerStarted","Data":"e78e3b6ed97011daded1402f544b3c05502e946a4a2b3946ab5de321d80dbdb1"}
Sep 10 12:09:58.272048 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:58.271992    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podStartSLOduration=1.271976822 podStartE2EDuration="1.271976822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:09:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:09:58.270095854 +0000 UTC m=+1108.873370771" watchObservedRunningTime="2026-09-10 12:09:58.271976822 +0000 UTC m=+1108.875251802"
Sep 10 12:09:58.441130 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:58.441088    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:58.445875 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:58.445849    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:59.254237 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:59.254202    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:59.255193 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:59.255172    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:09:59.714081 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:59.714051    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:59.845889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:59.845855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:09:59.948506 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:09:59.948479    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:00.108730 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:00.108627    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:00.254734 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:00.254699    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:00.355959 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:00.355926    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:01.390425 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:01.390393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:01.562751 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:01.562722    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:01.653587 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:01.653498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:01.877297 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:01.877265    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:02.041711 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.041677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:02.140542 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.140509    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:02.158379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.158350    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"]
Sep 10 12:10:02.161613 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.161596    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.164420 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.164399    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-c07be0-dockercfg-x7vqs\""
Sep 10 12:10:02.165029 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.165009    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-c07be0\""
Sep 10 12:10:02.178857 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.178826    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"]
Sep 10 12:10:02.283329 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.283292    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15223f0a-e868-4a11-93c8-aced51adeed2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.283502 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.283376    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wqhc\" (UniqueName: \"kubernetes.io/projected/15223f0a-e868-4a11-93c8-aced51adeed2-kube-api-access-2wqhc\") pod \"payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.384849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.384764    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wqhc\" (UniqueName: \"kubernetes.io/projected/15223f0a-e868-4a11-93c8-aced51adeed2-kube-api-access-2wqhc\") pod \"payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.384999 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.384848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15223f0a-e868-4a11-93c8-aced51adeed2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.385548 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.385523    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15223f0a-e868-4a11-93c8-aced51adeed2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.395629 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.395598    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wqhc\" (UniqueName: \"kubernetes.io/projected/15223f0a-e868-4a11-93c8-aced51adeed2-kube-api-access-2wqhc\") pod \"payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.472240 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.472204    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:02.744517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:02.744493    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"]
Sep 10 12:10:02.747147 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:10:02.747120    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15223f0a_e868_4a11_93c8_aced51adeed2.slice/crio-66706a31afc064fc1e417e371ba3620fe747f5386b05f902f9db32adcc9f8dab WatchSource:0}: Error finding container 66706a31afc064fc1e417e371ba3620fe747f5386b05f902f9db32adcc9f8dab: Status 404 returned error can't find the container with id 66706a31afc064fc1e417e371ba3620fe747f5386b05f902f9db32adcc9f8dab
Sep 10 12:10:03.277586 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:03.277548    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm" event={"ID":"15223f0a-e868-4a11-93c8-aced51adeed2","Type":"ContainerStarted","Data":"a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd"}
Sep 10 12:10:03.277777 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:03.277649    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:03.277777 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:03.277670    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm" event={"ID":"15223f0a-e868-4a11-93c8-aced51adeed2","Type":"ContainerStarted","Data":"66706a31afc064fc1e417e371ba3620fe747f5386b05f902f9db32adcc9f8dab"}
Sep 10 12:10:03.302960 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:03.302907    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm" podStartSLOduration=1.302891877 podStartE2EDuration="1.302891877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:03.300759233 +0000 UTC m=+1113.904034146" watchObservedRunningTime="2026-09-10 12:10:03.302891877 +0000 UTC m=+1113.906166863"
Sep 10 12:10:07.451987 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:07.451951    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:07.577014 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:07.576972    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:08.499563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.499529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:08.811930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.811839    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"]
Sep 10 12:10:08.821637 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.821609    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.828591 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.828570    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-639beb-openshift-default-dockercfg-4h9sd\""
Sep 10 12:10:08.846460 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.846435    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"]
Sep 10 12:10:08.866849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.866825    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:08.944589 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944551    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-certs\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944593    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-data\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944621    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfzsb\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-kube-api-access-mfzsb\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-token\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944881 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944768    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42117b39-84c6-4e4f-b99f-f1a934a8608c-istiod-ca-cert\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944881 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-podinfo\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944881 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944846    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-credential-socket\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.944881 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944870    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-envoy\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:08.945008 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:08.944935    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-socket\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.045936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.045904    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-socket\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.045936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.045935    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-certs\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046185 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.045951    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-data\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046185 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfzsb\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-kube-api-access-mfzsb\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046185 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046143    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-token\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046327 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046230    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42117b39-84c6-4e4f-b99f-f1a934a8608c-istiod-ca-cert\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046327 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046273    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-podinfo\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046327 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046314    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-credential-socket\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046487 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046351    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-envoy\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046487 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046380    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-certs\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046487 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046350    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-socket\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046655 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046600    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-credential-socket\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.046719 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046656    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-data\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.047011 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.046989    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42117b39-84c6-4e4f-b99f-f1a934a8608c-istiod-ca-cert\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.048469 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.048445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-envoy\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.048730 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.048713    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-podinfo\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.054857 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.054827    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfzsb\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-kube-api-access-mfzsb\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.055184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.055147    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-token\") pod \"e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") " pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.133406 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.133322    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:09.363034 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.362998    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:09.436803 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.436772    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"]
Sep 10 12:10:09.439447 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:10:09.439417    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42117b39_84c6_4e4f_b99f_f1a934a8608c.slice/crio-011009510ebf8e65aaaa5fc1a177003f3a7ccc0c0b02683e3aff9a2cbf4104a9 WatchSource:0}: Error finding container 011009510ebf8e65aaaa5fc1a177003f3a7ccc0c0b02683e3aff9a2cbf4104a9: Status 404 returned error can't find the container with id 011009510ebf8e65aaaa5fc1a177003f3a7ccc0c0b02683e3aff9a2cbf4104a9
Sep 10 12:10:09.441776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.441736    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:09.441868 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.441818    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:09.441868 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:09.441862    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:10.320949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:10.320913    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" event={"ID":"42117b39-84c6-4e4f-b99f-f1a934a8608c","Type":"ContainerStarted","Data":"36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"}
Sep 10 12:10:10.321372 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:10.320968    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" event={"ID":"42117b39-84c6-4e4f-b99f-f1a934a8608c","Type":"ContainerStarted","Data":"011009510ebf8e65aaaa5fc1a177003f3a7ccc0c0b02683e3aff9a2cbf4104a9"}
Sep 10 12:10:10.350740 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:10.350690    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podStartSLOduration=2.350675039 podStartE2EDuration="2.350675039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:10.349184781 +0000 UTC m=+1120.952459686" watchObservedRunningTime="2026-09-10 12:10:10.350675039 +0000 UTC m=+1120.953949949"
Sep 10 12:10:11.134033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:11.133992    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:11.138849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:11.138826    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:11.326763 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:11.326731    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:11.327860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:11.327837    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:12.397718 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:12.397679    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:12.519446 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:12.519410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:13.516927 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.516887    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"]
Sep 10 12:10:13.520649 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.520630    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.523286 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.523266    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-639beb-serving-cert\""
Sep 10 12:10:13.533784 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.533758    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"]
Sep 10 12:10:13.557260 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.557238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:13.589315 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.589286    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.589447 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.589320    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.589447 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.589349    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26xbs\" (UniqueName: \"kubernetes.io/projected/f3553857-7450-49b4-b247-5f1601862ae4-kube-api-access-26xbs\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.589447 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.589376    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3553857-7450-49b4-b247-5f1601862ae4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.660108 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.660077    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:13.690212 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.690177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.690365 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.690222    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.690365 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.690256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26xbs\" (UniqueName: \"kubernetes.io/projected/f3553857-7450-49b4-b247-5f1601862ae4-kube-api-access-26xbs\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.690365 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.690284    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3553857-7450-49b4-b247-5f1601862ae4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.691350 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.691313    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3553857-7450-49b4-b247-5f1601862ae4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.693050 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.693020    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.693136 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.693103    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.704256 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.704235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26xbs\" (UniqueName: \"kubernetes.io/projected/f3553857-7450-49b4-b247-5f1601862ae4-kube-api-access-26xbs\") pod \"maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:13.758678 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.758648    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:13.833106 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:13.833043    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:14.069426 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.069397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:14.127084 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.127053    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"]
Sep 10 12:10:14.128062 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:10:14.128030    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3553857_7450_49b4_b247_5f1601862ae4.slice/crio-5d3c1bb0bdef82eec871127ab6947933380ad8ff3826001814fbb39c4769ef09 WatchSource:0}: Error finding container 5d3c1bb0bdef82eec871127ab6947933380ad8ff3826001814fbb39c4769ef09: Status 404 returned error can't find the container with id 5d3c1bb0bdef82eec871127ab6947933380ad8ff3826001814fbb39c4769ef09
Sep 10 12:10:14.170368 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.170341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:14.285384 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.285350    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:10:14.347229 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.347186    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm" event={"ID":"f3553857-7450-49b4-b247-5f1601862ae4","Type":"ContainerStarted","Data":"5d3c1bb0bdef82eec871127ab6947933380ad8ff3826001814fbb39c4769ef09"}
Sep 10 12:10:14.924676 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.924641    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:60214->10.132.0.54:15021: read: connection reset by peer" start-of-body=
Sep 10 12:10:14.925106 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:14.924716    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": read tcp 10.132.0.2:60214->10.132.0.54:15021: read: connection reset by peer"
Sep 10 12:10:15.353800 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.353755    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm" event={"ID":"f3553857-7450-49b4-b247-5f1601862ae4","Type":"ContainerStarted","Data":"71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9"}
Sep 10 12:10:15.353991 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.353829    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:15.355327 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.355306    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerID="9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402" exitCode=0
Sep 10 12:10:15.355414 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.355337    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" event={"ID":"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a","Type":"ContainerDied","Data":"9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"}
Sep 10 12:10:15.355652 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.355640    2571 scope.go:117] "RemoveContainer" containerID="9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"
Sep 10 12:10:15.356666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.356640    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:15.356748 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.356734    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:15.356791 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.356771    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:15.376453 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:15.376412    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm" podStartSLOduration=2.058650387 podStartE2EDuration="2.376396949s" totalImagesPullingTime="317.746562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:15.375328114 +0000 UTC m=+1125.978603053" watchObservedRunningTime="2026-09-10 12:10:15.376396949 +0000 UTC m=+1125.979671859"
Sep 10 12:10:16.363304 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:16.363271    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" event={"ID":"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a","Type":"ContainerStarted","Data":"b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099"}
Sep 10 12:10:16.440564 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:16.440525    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:10:16.440807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:16.440782    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 10 12:10:16.440889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:16.440840    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 10 12:10:17.441197 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:17.441134    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 10 12:10:17.441585 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:17.441212    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 10 12:10:18.441309 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:18.441274    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 10 12:10:18.441701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:18.441328    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 10 12:10:19.441244 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:19.441204    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 10 12:10:19.441544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:19.441515    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 10 12:10:20.441794 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:20.441742    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 10 12:10:20.442209 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:20.441810    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 10 12:10:21.368538 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:21.368509    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:10:21.442273 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:21.442239    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 10 12:10:21.442730 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:21.442313    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 10 12:10:22.444968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:22.444937    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:10:22.445424 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:22.445244    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:10:22.445917 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:22.445899    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:10:22.574805 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:22.574777    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:25.205932 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:25.205899    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:25.558167 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:25.558113    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:25.929715 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:25.929674    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:26.353487 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:26.353399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:27.368314 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:27.368270    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 12:10:27.368731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:27.368334    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 12:10:28.444968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:28.444933    2571 generic.go:356] "Generic (PLEG): container finished" podID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerID="36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240" exitCode=0
Sep 10 12:10:28.445416 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:28.445004    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" event={"ID":"42117b39-84c6-4e4f-b99f-f1a934a8608c","Type":"ContainerDied","Data":"36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"}
Sep 10 12:10:28.445416 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:28.445404    2571 scope.go:117] "RemoveContainer" containerID="36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"
Sep 10 12:10:28.446392 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:28.446360    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:28.446505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:28.446446    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:28.446505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:28.446478    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:10:29.452521 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:29.452488    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" event={"ID":"42117b39-84c6-4e4f-b99f-f1a934a8608c","Type":"ContainerStarted","Data":"66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4"}
Sep 10 12:10:29.516935 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:29.516898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:30.134504 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:30.134467    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:30.134749 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:30.134728    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 12:10:30.134833 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:30.134781    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 12:10:30.139519 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:30.139494    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:30.738526 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:30.738492    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:30.837639 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:30.837608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:31.093363 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.093270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:31.133935 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.133901    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 12:10:31.134101 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.133958    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 12:10:31.487558 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.487520    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"]
Sep 10 12:10:31.491614 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.491587    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.494785 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.494758    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-7ed5ac\""
Sep 10 12:10:31.495004 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.494979    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-7ed5ac-dockercfg-5tsgg\""
Sep 10 12:10:31.509776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.509749    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"]
Sep 10 12:10:31.562379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.562345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.562556 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.562390    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9wxk\" (UniqueName: \"kubernetes.io/projected/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-kube-api-access-l9wxk\") pod \"payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.592198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.592144    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"]
Sep 10 12:10:31.595136 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.595109    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.611243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.611212    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"]
Sep 10 12:10:31.663468 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.663435    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.663468 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.663475    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9wxk\" (UniqueName: \"kubernetes.io/projected/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-kube-api-access-l9wxk\") pod \"payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.663706 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.663512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4ffd\" (UniqueName: \"kubernetes.io/projected/c247afe7-848d-4786-a345-03cff25809d4-kube-api-access-b4ffd\") pod \"payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.663706 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.663542    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c247afe7-848d-4786-a345-03cff25809d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.664229 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.664210    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.672131 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.672096    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9wxk\" (UniqueName: \"kubernetes.io/projected/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-kube-api-access-l9wxk\") pod \"payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.764623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.764518    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4ffd\" (UniqueName: \"kubernetes.io/projected/c247afe7-848d-4786-a345-03cff25809d4-kube-api-access-b4ffd\") pod \"payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.764623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.764572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c247afe7-848d-4786-a345-03cff25809d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.765304 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.765284    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c247afe7-848d-4786-a345-03cff25809d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.774280 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.774249    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4ffd\" (UniqueName: \"kubernetes.io/projected/c247afe7-848d-4786-a345-03cff25809d4-kube-api-access-b4ffd\") pod \"payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:31.807079 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.807039    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:31.832807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.832771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:31.912667 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:31.912629    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:32.103575 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.103460    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"]
Sep 10 12:10:32.106046 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:10:32.106020    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c151b8e_2ff0_4668_aa73_c7f992ca55fc.slice/crio-3bdb1913ee64789b7d7bc20a563712752d7dbf7156eb97eb52491ebc2aabf6f5 WatchSource:0}: Error finding container 3bdb1913ee64789b7d7bc20a563712752d7dbf7156eb97eb52491ebc2aabf6f5: Status 404 returned error can't find the container with id 3bdb1913ee64789b7d7bc20a563712752d7dbf7156eb97eb52491ebc2aabf6f5
Sep 10 12:10:32.134501 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.134475    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 12:10:32.134622 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.134531    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 12:10:32.194177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.193743    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"]
Sep 10 12:10:32.207408 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:10:32.207259    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc247afe7_848d_4786_a345_03cff25809d4.slice/crio-108fa73af4bbadabbc63a0e58bafa417fbae360d9760efe93584669bfc646eed WatchSource:0}: Error finding container 108fa73af4bbadabbc63a0e58bafa417fbae360d9760efe93584669bfc646eed: Status 404 returned error can't find the container with id 108fa73af4bbadabbc63a0e58bafa417fbae360d9760efe93584669bfc646eed
Sep 10 12:10:32.364677 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.364575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:32.468933 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.468897    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89" event={"ID":"9c151b8e-2ff0-4668-aa73-c7f992ca55fc","Type":"ContainerStarted","Data":"abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99"}
Sep 10 12:10:32.469144 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.468962    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:32.469144 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.468985    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89" event={"ID":"9c151b8e-2ff0-4668-aa73-c7f992ca55fc","Type":"ContainerStarted","Data":"3bdb1913ee64789b7d7bc20a563712752d7dbf7156eb97eb52491ebc2aabf6f5"}
Sep 10 12:10:32.470566 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.470539    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82" event={"ID":"c247afe7-848d-4786-a345-03cff25809d4","Type":"ContainerStarted","Data":"6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e"}
Sep 10 12:10:32.470690 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.470579    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82" event={"ID":"c247afe7-848d-4786-a345-03cff25809d4","Type":"ContainerStarted","Data":"108fa73af4bbadabbc63a0e58bafa417fbae360d9760efe93584669bfc646eed"}
Sep 10 12:10:32.470690 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.470610    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:32.522659 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.522605    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82" podStartSLOduration=1.522583918 podStartE2EDuration="1.522583918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:32.519357644 +0000 UTC m=+1143.122632556" watchObservedRunningTime="2026-09-10 12:10:32.522583918 +0000 UTC m=+1143.125858830"
Sep 10 12:10:32.523525 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:32.523475    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89" podStartSLOduration=1.523457839 podStartE2EDuration="1.523457839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:32.496724912 +0000 UTC m=+1143.099999824" watchObservedRunningTime="2026-09-10 12:10:32.523457839 +0000 UTC m=+1143.126732752"
Sep 10 12:10:33.133918 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:33.133879    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 12:10:33.134431 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:33.133944    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 12:10:34.133710 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:34.133678    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 12:10:34.133891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:34.133730    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 12:10:35.133773 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:35.133736    2571 patch_prober.go:28] interesting pod/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 10 12:10:35.134201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:35.133791    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 10 12:10:35.186303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:35.186267    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:36.137733 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:36.137701    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:36.138168 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:36.137973    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:36.138690 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:36.138672    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:10:37.374801 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:37.374775    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:37.710187 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:37.710137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:38.033992 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:38.033917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:38.353822 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:38.353751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:38.979876 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:38.979842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:41.549784 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:41.549752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:41.659468 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:41.659436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:42.744283 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:42.744247    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:42.914686 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:42.914657    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:43.444069 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.444037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:43.478958 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.478932    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:10:43.479143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.479007    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:10:43.556012 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.555974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:43.615691 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.615662    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"]
Sep 10 12:10:43.618551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.618535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.621407 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.621388    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-c66562-dockercfg-w9lw4\""
Sep 10 12:10:43.621575 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.621558    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-c66562\""
Sep 10 12:10:43.640419 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.640399    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"]
Sep 10 12:10:43.777603 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.777511    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5489a227-2ef3-4332-8b64-66a3de100725-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.777603 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.777580    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zp49\" (UniqueName: \"kubernetes.io/projected/5489a227-2ef3-4332-8b64-66a3de100725-kube-api-access-7zp49\") pod \"payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.878782 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.878739    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5489a227-2ef3-4332-8b64-66a3de100725-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.878981 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.878830    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zp49\" (UniqueName: \"kubernetes.io/projected/5489a227-2ef3-4332-8b64-66a3de100725-kube-api-access-7zp49\") pod \"payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.879517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.879491    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5489a227-2ef3-4332-8b64-66a3de100725-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.889398 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.889372    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zp49\" (UniqueName: \"kubernetes.io/projected/5489a227-2ef3-4332-8b64-66a3de100725-kube-api-access-7zp49\") pod \"payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:43.928609 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:43.928586    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:44.210765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:44.210733    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"]
Sep 10 12:10:44.211364 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:10:44.211339    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5489a227_2ef3_4332_8b64_66a3de100725.slice/crio-3ce9ba445dc6250ad42d2158292b72c1f6aeb203c461031457a2a553006ffe17 WatchSource:0}: Error finding container 3ce9ba445dc6250ad42d2158292b72c1f6aeb203c461031457a2a553006ffe17: Status 404 returned error can't find the container with id 3ce9ba445dc6250ad42d2158292b72c1f6aeb203c461031457a2a553006ffe17
Sep 10 12:10:44.549040 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:44.548934    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s" event={"ID":"5489a227-2ef3-4332-8b64-66a3de100725","Type":"ContainerStarted","Data":"5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c"}
Sep 10 12:10:44.549040 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:44.549030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s" event={"ID":"5489a227-2ef3-4332-8b64-66a3de100725","Type":"ContainerStarted","Data":"3ce9ba445dc6250ad42d2158292b72c1f6aeb203c461031457a2a553006ffe17"}
Sep 10 12:10:44.549303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:44.549084    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:10:44.587386 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:44.587332    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s" podStartSLOduration=1.587317895 podStartE2EDuration="1.587317895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:10:44.583347423 +0000 UTC m=+1155.186622336" watchObservedRunningTime="2026-09-10 12:10:44.587317895 +0000 UTC m=+1155.190592832"
Sep 10 12:10:51.777355 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:51.777314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:54.501339 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:54.501298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:10:55.556032 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:10:55.555996    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:11:05.331893 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:05.331849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:06.564946 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:06.564909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:21.667951 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:21.667921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:22.048291 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:22.048258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:22.288586 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:22.288551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:24.794068 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:24.794029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:24.888423 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:24.888387    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:25.936444 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:25.936410    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"]
Sep 10 12:11:25.941252 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:25.941234    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:25.943668 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:25.943645    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e74b3f66-serving-cert\""
Sep 10 12:11:25.951581 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:25.951556    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"]
Sep 10 12:11:26.040937 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.040909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:26.046184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.046140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbsz\" (UniqueName: \"kubernetes.io/projected/66388146-2ae5-486d-9278-cd6cf0d5c18d-kube-api-access-lqbsz\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.046308 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.046192    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.046402 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.046384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66388146-2ae5-486d-9278-cd6cf0d5c18d-openshift-service-ca\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.046447 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.046416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-metrics-certs\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.143952 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.143911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:26.147164 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.147126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66388146-2ae5-486d-9278-cd6cf0d5c18d-openshift-service-ca\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.147309 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.147177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-metrics-certs\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.147309 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.147235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbsz\" (UniqueName: \"kubernetes.io/projected/66388146-2ae5-486d-9278-cd6cf0d5c18d-kube-api-access-lqbsz\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.147309 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.147261    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.147475 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:11:26.147449    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-serving-cert: secret "maas-api-e2e-mt-e74b3f66-serving-cert" not found
Sep 10 12:11:26.147535 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:11:26.147522    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls podName:66388146-2ae5-486d-9278-cd6cf0d5c18d nodeName:}" failed. No retries permitted until 2026-09-10 12:11:26.647499948 +0000 UTC m=+1197.250774849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls") pod "maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" (UID: "66388146-2ae5-486d-9278-cd6cf0d5c18d") : secret "maas-api-e2e-mt-e74b3f66-serving-cert" not found
Sep 10 12:11:26.147875 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.147855    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66388146-2ae5-486d-9278-cd6cf0d5c18d-openshift-service-ca\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.149697 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.149678    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-metrics-certs\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.159625 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.159606    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbsz\" (UniqueName: \"kubernetes.io/projected/66388146-2ae5-486d-9278-cd6cf0d5c18d-kube-api-access-lqbsz\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.354855 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.354769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:26.552750 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.552709    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:26.646670 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.646583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:26.651522 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.651495    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.651802 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.651781    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"]
Sep 10 12:11:26.654128 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.654058    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls\") pod \"maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.655417 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.655400    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.657658 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.657638    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e74b3f66\""
Sep 10 12:11:26.657800 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.657720    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e74b3f66-dockercfg-7c5t8\""
Sep 10 12:11:26.667263 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.667239    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"]
Sep 10 12:11:26.752996 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.752965    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18b4c151-52d3-423a-b96a-ba611cff9e0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.752996 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.753005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp8xk\" (UniqueName: \"kubernetes.io/projected/18b4c151-52d3-423a-b96a-ba611cff9e0b-kube-api-access-cp8xk\") pod \"payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.851824 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.851792    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:26.854251 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.854224    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18b4c151-52d3-423a-b96a-ba611cff9e0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.854251 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.854252    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp8xk\" (UniqueName: \"kubernetes.io/projected/18b4c151-52d3-423a-b96a-ba611cff9e0b-kube-api-access-cp8xk\") pod \"payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.855007 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.854898    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18b4c151-52d3-423a-b96a-ba611cff9e0b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.864067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.864044    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp8xk\" (UniqueName: \"kubernetes.io/projected/18b4c151-52d3-423a-b96a-ba611cff9e0b-kube-api-access-cp8xk\") pod \"payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:26.972474 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:26.972441    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:27.127853 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.127814    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"]
Sep 10 12:11:27.128895 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:11:27.128869    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66388146_2ae5_486d_9278_cd6cf0d5c18d.slice/crio-6d0793a6dc473a0e453f459e15bd8025b05f9c800e170376d6aca61d3c37fff1 WatchSource:0}: Error finding container 6d0793a6dc473a0e453f459e15bd8025b05f9c800e170376d6aca61d3c37fff1: Status 404 returned error can't find the container with id 6d0793a6dc473a0e453f459e15bd8025b05f9c800e170376d6aca61d3c37fff1
Sep 10 12:11:27.245731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.245709    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"]
Sep 10 12:11:27.247718 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:11:27.247688    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b4c151_52d3_423a_b96a_ba611cff9e0b.slice/crio-4bfd062978dd330406f84093b56e2f04005dd99fd62c48d494c99f7eb4d1c957 WatchSource:0}: Error finding container 4bfd062978dd330406f84093b56e2f04005dd99fd62c48d494c99f7eb4d1c957: Status 404 returned error can't find the container with id 4bfd062978dd330406f84093b56e2f04005dd99fd62c48d494c99f7eb4d1c957
Sep 10 12:11:27.777505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.777468    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29" event={"ID":"18b4c151-52d3-423a-b96a-ba611cff9e0b","Type":"ContainerStarted","Data":"7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169"}
Sep 10 12:11:27.777716 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.777535    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29" event={"ID":"18b4c151-52d3-423a-b96a-ba611cff9e0b","Type":"ContainerStarted","Data":"4bfd062978dd330406f84093b56e2f04005dd99fd62c48d494c99f7eb4d1c957"}
Sep 10 12:11:27.777716 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.777565    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:27.778828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.778804    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" event={"ID":"66388146-2ae5-486d-9278-cd6cf0d5c18d","Type":"ContainerStarted","Data":"6d0793a6dc473a0e453f459e15bd8025b05f9c800e170376d6aca61d3c37fff1"}
Sep 10 12:11:27.796701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:27.796655    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29" podStartSLOduration=1.79663971 podStartE2EDuration="1.79663971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:27.794670326 +0000 UTC m=+1198.397945239" watchObservedRunningTime="2026-09-10 12:11:27.79663971 +0000 UTC m=+1198.399914620"
Sep 10 12:11:28.785705 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:28.785674    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" event={"ID":"66388146-2ae5-486d-9278-cd6cf0d5c18d","Type":"ContainerStarted","Data":"938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749"}
Sep 10 12:11:28.786116 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:28.785738    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:28.805528 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:28.805457    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" podStartSLOduration=2.7984511210000003 podStartE2EDuration="3.805433719s" totalImagesPullingTime="1.006982598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:11:28.803086234 +0000 UTC m=+1199.406361167" watchObservedRunningTime="2026-09-10 12:11:28.805433719 +0000 UTC m=+1199.408708634"
Sep 10 12:11:34.796123 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:34.796091    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:11:38.404994 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:38.404960    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:38.788637 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:38.788604    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:11:47.336985 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:47.336958    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:47.846141 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:47.846110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:11:49.784705 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:11:49.784672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:12:06.751936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.751904    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"]
Sep 10 12:12:06.755387 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.755367    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.758906 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.758881    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e74b3f66\"/\"openshift-service-ca.crt\""
Sep 10 12:12:06.758906 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.758901    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e74b3f66\"/\"kube-root-ca.crt\""
Sep 10 12:12:06.759115 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.758957    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e74b3f66\"/\"e2e-lifecycle-model-e74b3f66-kserve-self-signed-certs\""
Sep 10 12:12:06.759115 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.759025    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e74b3f66\"/\"default-dockercfg-vlhtf\""
Sep 10 12:12:06.764812 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.764773    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"]
Sep 10 12:12:06.776955 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.776935    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:12:06.804294 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.804270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-tmp-dir\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.804377 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.804333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9k9h\" (UniqueName: \"kubernetes.io/projected/07c34ff6-d980-429a-9f9f-08db664e6653-kube-api-access-d9k9h\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.804430 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.804381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c34ff6-d980-429a-9f9f-08db664e6653-tls-certs\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.804498 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.804477    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-home\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.804563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.804512    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-dshm\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.804563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.804540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-model-cache\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905144 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905120    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9k9h\" (UniqueName: \"kubernetes.io/projected/07c34ff6-d980-429a-9f9f-08db664e6653-kube-api-access-d9k9h\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905302 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905186    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c34ff6-d980-429a-9f9f-08db664e6653-tls-certs\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905302 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905229    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-home\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905302 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905244    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-dshm\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905302 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905260    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-model-cache\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905302 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-tmp-dir\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905621 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905594    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-home\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905729 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905640    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-model-cache\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.905729 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.905716    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-tmp-dir\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.907581 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.907555    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-dshm\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.907828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.907812    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c34ff6-d980-429a-9f9f-08db664e6653-tls-certs\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:06.914468 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:06.914449    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9k9h\" (UniqueName: \"kubernetes.io/projected/07c34ff6-d980-429a-9f9f-08db664e6653-kube-api-access-d9k9h\") pod \"e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") " pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:07.065539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:07.065446    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:07.340246 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:07.340218    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"]
Sep 10 12:12:07.342135 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:12:07.342104    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07c34ff6_d980_429a_9f9f_08db664e6653.slice/crio-f7597ac88df8dcaf862d07c966365bd3eab0223da09752cf869eff9322316b95 WatchSource:0}: Error finding container f7597ac88df8dcaf862d07c966365bd3eab0223da09752cf869eff9322316b95: Status 404 returned error can't find the container with id f7597ac88df8dcaf862d07c966365bd3eab0223da09752cf869eff9322316b95
Sep 10 12:12:07.996992 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:07.996953    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj" event={"ID":"07c34ff6-d980-429a-9f9f-08db664e6653","Type":"ContainerStarted","Data":"1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad"}
Sep 10 12:12:07.997377 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:07.997003    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj" event={"ID":"07c34ff6-d980-429a-9f9f-08db664e6653","Type":"ContainerStarted","Data":"f7597ac88df8dcaf862d07c966365bd3eab0223da09752cf869eff9322316b95"}
Sep 10 12:12:08.014823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:08.014772    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj" podStartSLOduration=2.014752091 podStartE2EDuration="2.014752091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:12:08.013938134 +0000 UTC m=+1238.617213046" watchObservedRunningTime="2026-09-10 12:12:08.014752091 +0000 UTC m=+1238.618027002"
Sep 10 12:12:17.065836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:17.065801    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:17.065836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:17.065845    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:17.078481 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:17.078456    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:18.061080 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:18.061051    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:12:24.699111 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:12:24.699042    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:13:16.254016 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.253966    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"]
Sep 10 12:13:16.254559 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.254262    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" podUID="66388146-2ae5-486d-9278-cd6cf0d5c18d" containerName="maas-api" containerID="cri-o://938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749" gracePeriod=30
Sep 10 12:13:16.288380 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.288352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:13:16.315768 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.315736    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"]
Sep 10 12:13:16.316015 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.315968    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29" podUID="18b4c151-52d3-423a-b96a-ba611cff9e0b" containerName="payload-pre-processing" containerID="cri-o://7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169" gracePeriod=30
Sep 10 12:13:16.574059 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.574025    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:13:16.610823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.610800    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:13:16.727379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727348    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66388146-2ae5-486d-9278-cd6cf0d5c18d-openshift-service-ca\") pod \"66388146-2ae5-486d-9278-cd6cf0d5c18d\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") "
Sep 10 12:13:16.727379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727378    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18b4c151-52d3-423a-b96a-ba611cff9e0b-plugins-config-volume\") pod \"18b4c151-52d3-423a-b96a-ba611cff9e0b\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") "
Sep 10 12:13:16.727619 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727406    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-metrics-certs\") pod \"66388146-2ae5-486d-9278-cd6cf0d5c18d\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") "
Sep 10 12:13:16.727619 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727553    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls\") pod \"66388146-2ae5-486d-9278-cd6cf0d5c18d\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") "
Sep 10 12:13:16.727619 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727604    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp8xk\" (UniqueName: \"kubernetes.io/projected/18b4c151-52d3-423a-b96a-ba611cff9e0b-kube-api-access-cp8xk\") pod \"18b4c151-52d3-423a-b96a-ba611cff9e0b\" (UID: \"18b4c151-52d3-423a-b96a-ba611cff9e0b\") "
Sep 10 12:13:16.727772 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727665    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqbsz\" (UniqueName: \"kubernetes.io/projected/66388146-2ae5-486d-9278-cd6cf0d5c18d-kube-api-access-lqbsz\") pod \"66388146-2ae5-486d-9278-cd6cf0d5c18d\" (UID: \"66388146-2ae5-486d-9278-cd6cf0d5c18d\") "
Sep 10 12:13:16.727822 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727774    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66388146-2ae5-486d-9278-cd6cf0d5c18d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66388146-2ae5-486d-9278-cd6cf0d5c18d" (UID: "66388146-2ae5-486d-9278-cd6cf0d5c18d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:13:16.727822 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727790    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18b4c151-52d3-423a-b96a-ba611cff9e0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18b4c151-52d3-423a-b96a-ba611cff9e0b" (UID: "18b4c151-52d3-423a-b96a-ba611cff9e0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:13:16.728020 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.727992    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66388146-2ae5-486d-9278-cd6cf0d5c18d-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:16.728075 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.728026    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18b4c151-52d3-423a-b96a-ba611cff9e0b-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:16.729760 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.729729    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66388146-2ae5-486d-9278-cd6cf0d5c18d" (UID: "66388146-2ae5-486d-9278-cd6cf0d5c18d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:13:16.729891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.729811    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "66388146-2ae5-486d-9278-cd6cf0d5c18d" (UID: "66388146-2ae5-486d-9278-cd6cf0d5c18d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:13:16.729957 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.729912    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18b4c151-52d3-423a-b96a-ba611cff9e0b-kube-api-access-cp8xk" (OuterVolumeSpecName: "kube-api-access-cp8xk") pod "18b4c151-52d3-423a-b96a-ba611cff9e0b" (UID: "18b4c151-52d3-423a-b96a-ba611cff9e0b"). InnerVolumeSpecName "kube-api-access-cp8xk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:13:16.730212 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.730192    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66388146-2ae5-486d-9278-cd6cf0d5c18d-kube-api-access-lqbsz" (OuterVolumeSpecName: "kube-api-access-lqbsz") pod "66388146-2ae5-486d-9278-cd6cf0d5c18d" (UID: "66388146-2ae5-486d-9278-cd6cf0d5c18d"). InnerVolumeSpecName "kube-api-access-lqbsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:13:16.829340 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.829256    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:16.829340 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.829282    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66388146-2ae5-486d-9278-cd6cf0d5c18d-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:16.829340 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.829291    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp8xk\" (UniqueName: \"kubernetes.io/projected/18b4c151-52d3-423a-b96a-ba611cff9e0b-kube-api-access-cp8xk\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:16.829340 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:16.829300    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqbsz\" (UniqueName: \"kubernetes.io/projected/66388146-2ae5-486d-9278-cd6cf0d5c18d-kube-api-access-lqbsz\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:17.366790 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.366755    2571 generic.go:356] "Generic (PLEG): container finished" podID="18b4c151-52d3-423a-b96a-ba611cff9e0b" containerID="7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169" exitCode=0
Sep 10 12:13:17.367270 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.366800    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"
Sep 10 12:13:17.367270 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.366840    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29" event={"ID":"18b4c151-52d3-423a-b96a-ba611cff9e0b","Type":"ContainerDied","Data":"7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169"}
Sep 10 12:13:17.367270 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.366894    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29" event={"ID":"18b4c151-52d3-423a-b96a-ba611cff9e0b","Type":"ContainerDied","Data":"4bfd062978dd330406f84093b56e2f04005dd99fd62c48d494c99f7eb4d1c957"}
Sep 10 12:13:17.367270 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.366917    2571 scope.go:117] "RemoveContainer" containerID="7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169"
Sep 10 12:13:17.368415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.368207    2571 generic.go:356] "Generic (PLEG): container finished" podID="66388146-2ae5-486d-9278-cd6cf0d5c18d" containerID="938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749" exitCode=0
Sep 10 12:13:17.368415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.368250    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"
Sep 10 12:13:17.368415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.368302    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" event={"ID":"66388146-2ae5-486d-9278-cd6cf0d5c18d","Type":"ContainerDied","Data":"938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749"}
Sep 10 12:13:17.368415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.368357    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s" event={"ID":"66388146-2ae5-486d-9278-cd6cf0d5c18d","Type":"ContainerDied","Data":"6d0793a6dc473a0e453f459e15bd8025b05f9c800e170376d6aca61d3c37fff1"}
Sep 10 12:13:17.378470 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.378448    2571 scope.go:117] "RemoveContainer" containerID="7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169"
Sep 10 12:13:17.378762 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:13:17.378742    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169\": container with ID starting with 7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169 not found: ID does not exist" containerID="7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169"
Sep 10 12:13:17.378825 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.378770    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169"} err="failed to get container status \"7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169\": rpc error: code = NotFound desc = could not find container \"7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169\": container with ID starting with 7b78573e6ebb775cec721bd285b8bc05c803bf1088f1e1b204792d01d4043169 not found: ID does not exist"
Sep 10 12:13:17.378825 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.378787    2571 scope.go:117] "RemoveContainer" containerID="938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749"
Sep 10 12:13:17.386609 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.386590    2571 scope.go:117] "RemoveContainer" containerID="938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749"
Sep 10 12:13:17.386847 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:13:17.386831    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749\": container with ID starting with 938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749 not found: ID does not exist" containerID="938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749"
Sep 10 12:13:17.386905 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.386852    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749"} err="failed to get container status \"938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749\": rpc error: code = NotFound desc = could not find container \"938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749\": container with ID starting with 938aab225e359dda9229fd12d5c6e4367284a7d12b95501c5024ae919b893749 not found: ID does not exist"
Sep 10 12:13:17.389897 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.389875    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"]
Sep 10 12:13:17.394442 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.394420    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e74b3f66-5b644d9b46-kdd29"]
Sep 10 12:13:17.404832 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.404808    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"]
Sep 10 12:13:17.409312 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.409292    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e74b3f66-5b99fc5875-d8f2s"]
Sep 10 12:13:17.892468 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.892437    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18b4c151-52d3-423a-b96a-ba611cff9e0b" path="/var/lib/kubelet/pods/18b4c151-52d3-423a-b96a-ba611cff9e0b/volumes"
Sep 10 12:13:17.892849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:17.892835    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66388146-2ae5-486d-9278-cd6cf0d5c18d" path="/var/lib/kubelet/pods/66388146-2ae5-486d-9278-cd6cf0d5c18d/volumes"
Sep 10 12:13:18.034419 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:18.034386    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:13:20.034652 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:20.034621    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:13:25.020277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:25.020240    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"]
Sep 10 12:13:25.118255 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:25.118228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:13:40.039911 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.039856    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj" podUID="07c34ff6-d980-429a-9f9f-08db664e6653" containerName="main" containerID="cri-o://1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad" gracePeriod=45
Sep 10 12:13:40.285213 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.285190    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:13:40.334071 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.333982    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-model-cache\") pod \"07c34ff6-d980-429a-9f9f-08db664e6653\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") "
Sep 10 12:13:40.334267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334100    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-tmp-dir\") pod \"07c34ff6-d980-429a-9f9f-08db664e6653\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") "
Sep 10 12:13:40.334267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334165    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9k9h\" (UniqueName: \"kubernetes.io/projected/07c34ff6-d980-429a-9f9f-08db664e6653-kube-api-access-d9k9h\") pod \"07c34ff6-d980-429a-9f9f-08db664e6653\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") "
Sep 10 12:13:40.334267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334195    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c34ff6-d980-429a-9f9f-08db664e6653-tls-certs\") pod \"07c34ff6-d980-429a-9f9f-08db664e6653\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") "
Sep 10 12:13:40.334267 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334226    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-home\") pod \"07c34ff6-d980-429a-9f9f-08db664e6653\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") "
Sep 10 12:13:40.334505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334282    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-dshm\") pod \"07c34ff6-d980-429a-9f9f-08db664e6653\" (UID: \"07c34ff6-d980-429a-9f9f-08db664e6653\") "
Sep 10 12:13:40.334505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334285    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-model-cache" (OuterVolumeSpecName: "model-cache") pod "07c34ff6-d980-429a-9f9f-08db664e6653" (UID: "07c34ff6-d980-429a-9f9f-08db664e6653"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:13:40.334505 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334395    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07c34ff6-d980-429a-9f9f-08db664e6653" (UID: "07c34ff6-d980-429a-9f9f-08db664e6653"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:13:40.334681 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334567    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-tmp-dir\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:40.334681 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334587    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-home" (OuterVolumeSpecName: "home") pod "07c34ff6-d980-429a-9f9f-08db664e6653" (UID: "07c34ff6-d980-429a-9f9f-08db664e6653"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:13:40.334681 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.334586    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-model-cache\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:40.336477 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.336441    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07c34ff6-d980-429a-9f9f-08db664e6653-kube-api-access-d9k9h" (OuterVolumeSpecName: "kube-api-access-d9k9h") pod "07c34ff6-d980-429a-9f9f-08db664e6653" (UID: "07c34ff6-d980-429a-9f9f-08db664e6653"). InnerVolumeSpecName "kube-api-access-d9k9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:13:40.336588 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.336494    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07c34ff6-d980-429a-9f9f-08db664e6653-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07c34ff6-d980-429a-9f9f-08db664e6653" (UID: "07c34ff6-d980-429a-9f9f-08db664e6653"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:13:40.336760 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.336741    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-dshm" (OuterVolumeSpecName: "dshm") pod "07c34ff6-d980-429a-9f9f-08db664e6653" (UID: "07c34ff6-d980-429a-9f9f-08db664e6653"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:13:40.435399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.435361    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9k9h\" (UniqueName: \"kubernetes.io/projected/07c34ff6-d980-429a-9f9f-08db664e6653-kube-api-access-d9k9h\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:40.435399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.435395    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07c34ff6-d980-429a-9f9f-08db664e6653-tls-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:40.435599 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.435409    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-home\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:40.435599 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.435420    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07c34ff6-d980-429a-9f9f-08db664e6653-dshm\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:13:40.492708 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.492673    2571 generic.go:356] "Generic (PLEG): container finished" podID="07c34ff6-d980-429a-9f9f-08db664e6653" containerID="1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad" exitCode=0
Sep 10 12:13:40.492708 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.492711    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"
Sep 10 12:13:40.492946 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.492760    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj" event={"ID":"07c34ff6-d980-429a-9f9f-08db664e6653","Type":"ContainerDied","Data":"1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad"}
Sep 10 12:13:40.492946 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.492806    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj" event={"ID":"07c34ff6-d980-429a-9f9f-08db664e6653","Type":"ContainerDied","Data":"f7597ac88df8dcaf862d07c966365bd3eab0223da09752cf869eff9322316b95"}
Sep 10 12:13:40.492946 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.492828    2571 scope.go:117] "RemoveContainer" containerID="1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad"
Sep 10 12:13:40.501622 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.501606    2571 scope.go:117] "RemoveContainer" containerID="1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad"
Sep 10 12:13:40.501897 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:13:40.501879    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad\": container with ID starting with 1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad not found: ID does not exist" containerID="1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad"
Sep 10 12:13:40.501968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.501908    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad"} err="failed to get container status \"1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad\": rpc error: code = NotFound desc = could not find container \"1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad\": container with ID starting with 1aacdabec94b738c93687c5cd78596276a889026630d9683aacc5bf4ca206cad not found: ID does not exist"
Sep 10 12:13:40.513902 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.513878    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"]
Sep 10 12:13:40.519828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:40.519803    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e74b3f66/e2e-lifecycle-model-e74b3f66-kserve-fcd889c5-7cpsj"]
Sep 10 12:13:41.241934 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:41.241899    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:13:41.892007 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:13:41.891970    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07c34ff6-d980-429a-9f9f-08db664e6653" path="/var/lib/kubelet/pods/07c34ff6-d980-429a-9f9f-08db664e6653/volumes"
Sep 10 12:14:01.161424 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:01.161394    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:02.328406 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:02.328367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:07.073474 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:07.073440    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:08.938238 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:08.938205    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:14.720699 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:14.720663    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:14.815362 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:14.815326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:14.980325 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:14.980240    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:16.301513 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:16.301477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:16.483445 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:16.483410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:16.579891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:16.579812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:17.171378 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:17.171340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:17.290391 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:17.290360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:18.689921 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:18.689887    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:25.786871 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:25.786826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:25.890672 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:25.890639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:27.334288 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:27.334254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:32.470252 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:32.470219    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:32.585840 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:32.585801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:32.682273 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:32.682238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:43.257671 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:43.257635    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:43.489297 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:43.489262    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:45.929223 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:45.929180    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:46.900781 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:46.900751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:54.085720 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.085686    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"]
Sep 10 12:14:54.086121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086034    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66388146-2ae5-486d-9278-cd6cf0d5c18d" containerName="maas-api"
Sep 10 12:14:54.086121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086045    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="66388146-2ae5-486d-9278-cd6cf0d5c18d" containerName="maas-api"
Sep 10 12:14:54.086121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086060    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18b4c151-52d3-423a-b96a-ba611cff9e0b" containerName="payload-pre-processing"
Sep 10 12:14:54.086121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086066    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="18b4c151-52d3-423a-b96a-ba611cff9e0b" containerName="payload-pre-processing"
Sep 10 12:14:54.086121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086076    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07c34ff6-d980-429a-9f9f-08db664e6653" containerName="main"
Sep 10 12:14:54.086121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086081    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="07c34ff6-d980-429a-9f9f-08db664e6653" containerName="main"
Sep 10 12:14:54.086353 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086136    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="66388146-2ae5-486d-9278-cd6cf0d5c18d" containerName="maas-api"
Sep 10 12:14:54.086353 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086145    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="07c34ff6-d980-429a-9f9f-08db664e6653" containerName="main"
Sep 10 12:14:54.086353 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.086164    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="18b4c151-52d3-423a-b96a-ba611cff9e0b" containerName="payload-pre-processing"
Sep 10 12:14:54.090285 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.090267    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.094596 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.094568    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c07be0\"/\"default-dockercfg-2gvdn\""
Sep 10 12:14:54.094814 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.094795    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-c07be0\"/\"kube-root-ca.crt\""
Sep 10 12:14:54.095019 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.095002    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-c07be0\"/\"openshift-service-ca.crt\""
Sep 10 12:14:54.095554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.095538    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-c07be0\"/\"sub-test-model-c07be0-kserve-self-signed-certs\""
Sep 10 12:14:54.116492 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.116456    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"]
Sep 10 12:14:54.172669 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.172634    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aedb7fc-5321-4772-9f8f-255575cc09c4-tls-certs\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.172850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.172677    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-home\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.172850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.172749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-dshm\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.172850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.172770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-model-cache\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.172850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.172793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-tmp-dir\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.172850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.172814    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6rsm\" (UniqueName: \"kubernetes.io/projected/0aedb7fc-5321-4772-9f8f-255575cc09c4-kube-api-access-r6rsm\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.187750 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.187723    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:14:54.273561 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.273524    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-dshm\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.273561 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.273561    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-model-cache\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.273807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.273578    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-tmp-dir\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.273807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.273594    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6rsm\" (UniqueName: \"kubernetes.io/projected/0aedb7fc-5321-4772-9f8f-255575cc09c4-kube-api-access-r6rsm\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.273807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.273668    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aedb7fc-5321-4772-9f8f-255575cc09c4-tls-certs\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.273807 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.273690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-home\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.274043 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.274017    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-model-cache\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.274108 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.274059    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-tmp-dir\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.274108 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.274091    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-home\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.275911 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.275894    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-dshm\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.276227 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.276206    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aedb7fc-5321-4772-9f8f-255575cc09c4-tls-certs\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.285240 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.285215    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6rsm\" (UniqueName: \"kubernetes.io/projected/0aedb7fc-5321-4772-9f8f-255575cc09c4-kube-api-access-r6rsm\") pod \"sub-test-model-c07be0-kserve-5696d6665-9bv28\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") " pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.400238 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.400137    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:14:54.675737 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.675707    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"]
Sep 10 12:14:54.677599 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:14:54.677571    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aedb7fc_5321_4772_9f8f_255575cc09c4.slice/crio-1d757698f988b4ca40bfd31b99734c9d60444813712290869fe38abc7432a575 WatchSource:0}: Error finding container 1d757698f988b4ca40bfd31b99734c9d60444813712290869fe38abc7432a575: Status 404 returned error can't find the container with id 1d757698f988b4ca40bfd31b99734c9d60444813712290869fe38abc7432a575
Sep 10 12:14:54.885570 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.885528    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28" event={"ID":"0aedb7fc-5321-4772-9f8f-255575cc09c4","Type":"ContainerStarted","Data":"0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5"}
Sep 10 12:14:54.885570 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.885573    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28" event={"ID":"0aedb7fc-5321-4772-9f8f-255575cc09c4","Type":"ContainerStarted","Data":"1d757698f988b4ca40bfd31b99734c9d60444813712290869fe38abc7432a575"}
Sep 10 12:14:54.908751 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:14:54.908696    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28" podStartSLOduration=0.908682042 podStartE2EDuration="908.682042ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:14:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:14:54.905837742 +0000 UTC m=+1405.509112654" watchObservedRunningTime="2026-09-10 12:14:54.908682042 +0000 UTC m=+1405.511956976"
Sep 10 12:15:00.160858 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.160823    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"]
Sep 10 12:15:00.164080 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.164064    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"
Sep 10 12:15:00.167205 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.167183    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-j7ls2\""
Sep 10 12:15:00.173907 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.173886    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"
Sep 10 12:15:00.182798 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.182767    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"]
Sep 10 12:15:00.453227 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.453202    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"]
Sep 10 12:15:00.454362 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:15:00.454341    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948fafd0_3252_46d1_8466_5aea42bc4dc6.slice/crio-91dd680b5789459109679d560eb83f9f2cca276407ea27f422a3b94cf4157090 WatchSource:0}: Error finding container 91dd680b5789459109679d560eb83f9f2cca276407ea27f422a3b94cf4157090: Status 404 returned error can't find the container with id 91dd680b5789459109679d560eb83f9f2cca276407ea27f422a3b94cf4157090
Sep 10 12:15:00.456199 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.456183    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:15:00.919835 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:00.919753    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl" event={"ID":"948fafd0-3252-46d1-8466-5aea42bc4dc6","Type":"ContainerStarted","Data":"91dd680b5789459109679d560eb83f9f2cca276407ea27f422a3b94cf4157090"}
Sep 10 12:15:03.937752 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:03.937719    2571 generic.go:356] "Generic (PLEG): container finished" podID="948fafd0-3252-46d1-8466-5aea42bc4dc6" containerID="ce70579b369f4df2c786acdd19d2d27d2e7abb184ad28f30520b325bcddff1e8" exitCode=0
Sep 10 12:15:03.938140 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:03.937801    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl" event={"ID":"948fafd0-3252-46d1-8466-5aea42bc4dc6","Type":"ContainerDied","Data":"ce70579b369f4df2c786acdd19d2d27d2e7abb184ad28f30520b325bcddff1e8"}
Sep 10 12:15:04.400698 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:04.400602    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:15:04.400698 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:04.400654    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:15:04.413508 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:04.413476    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:15:04.462324 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:04.462288    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:04.955024 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:04.954994    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:15:05.080544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:05.080514    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"
Sep 10 12:15:05.949444 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:05.949416    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"
Sep 10 12:15:05.949606 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:05.949416    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl" event={"ID":"948fafd0-3252-46d1-8466-5aea42bc4dc6","Type":"ContainerDied","Data":"91dd680b5789459109679d560eb83f9f2cca276407ea27f422a3b94cf4157090"}
Sep 10 12:15:05.949606 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:05.949535    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91dd680b5789459109679d560eb83f9f2cca276407ea27f422a3b94cf4157090"
Sep 10 12:15:10.270144 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:10.270110    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:10.347669 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:10.347632    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:10.606678 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:10.606606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:10.938864 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:10.938835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:11.350344 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:11.350269    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:14.382902 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:14.382855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:14.609264 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:14.609235    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:15.451369 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:15.451341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:15.554814 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:15.554783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:15.991853 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:15.991818    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:16.160277 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.160242    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:16.260087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.260008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:16.296331 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.296300    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"]
Sep 10 12:15:16.296710 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.296693    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948fafd0-3252-46d1-8466-5aea42bc4dc6" containerName="cleanup"
Sep 10 12:15:16.296796 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.296712    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="948fafd0-3252-46d1-8466-5aea42bc4dc6" containerName="cleanup"
Sep 10 12:15:16.296853 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.296829    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="948fafd0-3252-46d1-8466-5aea42bc4dc6" containerName="cleanup"
Sep 10 12:15:16.299871 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.299850    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.302963 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.302943    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-187e7e\""
Sep 10 12:15:16.302963 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.302953    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-187e7e-dockercfg-n7zxf\""
Sep 10 12:15:16.313922 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.313898    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"]
Sep 10 12:15:16.377544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.377479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7grn2\" (UniqueName: \"kubernetes.io/projected/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-kube-api-access-7grn2\") pod \"payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.377703 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.377600    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.388629 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.388596    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"]
Sep 10 12:15:16.392906 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.392882    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.405936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.405900    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"]
Sep 10 12:15:16.478750 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.478718    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/598b85a4-200b-4aa5-9b6c-743dac6a09fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.479247 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.478759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7grn2\" (UniqueName: \"kubernetes.io/projected/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-kube-api-access-7grn2\") pod \"payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.479247 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.478802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.479247 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.478822    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vz59\" (UniqueName: \"kubernetes.io/projected/598b85a4-200b-4aa5-9b6c-743dac6a09fd-kube-api-access-8vz59\") pod \"payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.479484 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.479464    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.488344 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.488321    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7grn2\" (UniqueName: \"kubernetes.io/projected/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-kube-api-access-7grn2\") pod \"payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.579607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.579523    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/598b85a4-200b-4aa5-9b6c-743dac6a09fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.579607 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.579592    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vz59\" (UniqueName: \"kubernetes.io/projected/598b85a4-200b-4aa5-9b6c-743dac6a09fd-kube-api-access-8vz59\") pod \"payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.580320 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.580298    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/598b85a4-200b-4aa5-9b6c-743dac6a09fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.589362 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.589340    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vz59\" (UniqueName: \"kubernetes.io/projected/598b85a4-200b-4aa5-9b6c-743dac6a09fd-kube-api-access-8vz59\") pod \"payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.609814 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.609796    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:16.708911 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.708884    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:16.891572 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.891543    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"]
Sep 10 12:15:16.893214 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:15:16.893191    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adaa6ef_2f81_479e_9bb4_e8e6c8dd5cce.slice/crio-7a83576ee68849f85bae56ed32c1f0f04617f90269890e703e012275c516ccc0 WatchSource:0}: Error finding container 7a83576ee68849f85bae56ed32c1f0f04617f90269890e703e012275c516ccc0: Status 404 returned error can't find the container with id 7a83576ee68849f85bae56ed32c1f0f04617f90269890e703e012275c516ccc0
Sep 10 12:15:16.984037 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:16.984010    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"]
Sep 10 12:15:16.984173 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:15:16.984033    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598b85a4_200b_4aa5_9b6c_743dac6a09fd.slice/crio-dba76ecb1fc249fb600dcf67fea6645cb7d28c68ed5f7e19abc2c5de9dbb05f4 WatchSource:0}: Error finding container dba76ecb1fc249fb600dcf67fea6645cb7d28c68ed5f7e19abc2c5de9dbb05f4: Status 404 returned error can't find the container with id dba76ecb1fc249fb600dcf67fea6645cb7d28c68ed5f7e19abc2c5de9dbb05f4
Sep 10 12:15:17.010046 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:17.010008    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6" event={"ID":"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce","Type":"ContainerStarted","Data":"5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6"}
Sep 10 12:15:17.010189 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:17.010080    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:17.010189 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:17.010106    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6" event={"ID":"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce","Type":"ContainerStarted","Data":"7a83576ee68849f85bae56ed32c1f0f04617f90269890e703e012275c516ccc0"}
Sep 10 12:15:17.011518 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:17.011496    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f" event={"ID":"598b85a4-200b-4aa5-9b6c-743dac6a09fd","Type":"ContainerStarted","Data":"dba76ecb1fc249fb600dcf67fea6645cb7d28c68ed5f7e19abc2c5de9dbb05f4"}
Sep 10 12:15:17.042871 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:17.042827    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6" podStartSLOduration=1.0428115550000001 podStartE2EDuration="1.042811555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:17.041938187 +0000 UTC m=+1427.645213098" watchObservedRunningTime="2026-09-10 12:15:17.042811555 +0000 UTC m=+1427.646086466"
Sep 10 12:15:18.019797 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:18.019762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f" event={"ID":"598b85a4-200b-4aa5-9b6c-743dac6a09fd","Type":"ContainerStarted","Data":"2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f"}
Sep 10 12:15:18.020287 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:18.020147    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:18.054363 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:18.054313    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f" podStartSLOduration=2.054298198 podStartE2EDuration="2.054298198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:18.052170347 +0000 UTC m=+1428.655445257" watchObservedRunningTime="2026-09-10 12:15:18.054298198 +0000 UTC m=+1428.657573108"
Sep 10 12:15:18.853801 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:18.853769    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:19.004013 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:19.003974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:23.705011 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:23.704932    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:28.021845 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:28.021815    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:15:29.027999 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:29.027958    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:15:35.544938 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:35.544902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:36.593475 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:36.593446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:37.752333 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:37.752301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:39.290452 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:39.290409    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:39.804434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:39.804403    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:40.273419 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:40.273384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:40.481120 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:40.481091    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:43.363453 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:43.363410    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:43.453729 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:43.453691    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:44.557595 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.557559    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"]
Sep 10 12:15:44.565119 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.565098    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.567948 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.567923    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c531cb-serving-cert\""
Sep 10 12:15:44.575551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.575526    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"]
Sep 10 12:15:44.596654 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.596621    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:44.640971 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.640942    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-metrics-certs\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.641113 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.640975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.641113 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.641060    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-openshift-service-ca\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.641226 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.641113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l5zq\" (UniqueName: \"kubernetes.io/projected/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-kube-api-access-6l5zq\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.671177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.671122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:44.742382 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.742352    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-openshift-service-ca\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.742582 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.742397    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l5zq\" (UniqueName: \"kubernetes.io/projected/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-kube-api-access-6l5zq\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.742582 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.742450    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-metrics-certs\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.742582 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.742471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.742582 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:15:44.742573    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-serving-cert: secret "maas-api-e2e-api-b-c531cb-serving-cert" not found
Sep 10 12:15:44.742838 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:15:44.742634    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls podName:92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f nodeName:}" failed. No retries permitted until 2026-09-10 12:15:45.242613685 +0000 UTC m=+1455.845888576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls") pod "maas-api-e2e-api-b-c531cb-7b64598947-k46lp" (UID: "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f") : secret "maas-api-e2e-api-b-c531cb-serving-cert" not found
Sep 10 12:15:44.743145 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.743115    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-openshift-service-ca\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.745038 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.745018    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-metrics-certs\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.753677 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.753652    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l5zq\" (UniqueName: \"kubernetes.io/projected/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-kube-api-access-6l5zq\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:44.821265 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:44.821190    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:45.152795 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.152714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:45.243580 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.243543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:45.246483 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.246457    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:45.249351 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.249324    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls\") pod \"maas-api-e2e-api-b-c531cb-7b64598947-k46lp\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:45.268937 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.268906    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"]
Sep 10 12:15:45.272735 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.272719    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.275177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.275142    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c531cb-dockercfg-7dg6c\""
Sep 10 12:15:45.275279 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.275176    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c531cb\""
Sep 10 12:15:45.282950 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.282922    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"]
Sep 10 12:15:45.347256 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.347216    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf4w2\" (UniqueName: \"kubernetes.io/projected/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-kube-api-access-cf4w2\") pod \"payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.347470 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.347267    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.448555 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.448522    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf4w2\" (UniqueName: \"kubernetes.io/projected/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-kube-api-access-cf4w2\") pod \"payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.448555 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.448558    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.449241 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.449223    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.457033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.457011    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf4w2\" (UniqueName: \"kubernetes.io/projected/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-kube-api-access-cf4w2\") pod \"payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.477567 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.477542    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:45.583949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.583922    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:45.760669 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.760642    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"]
Sep 10 12:15:45.762735 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:15:45.762704    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92efd4e4_c1f6_4dc4_a0fc_ad5ab36b8d8f.slice/crio-f837153e2c2410be305fccfc5d0a0c9f41401bddfafba7aa24e0b4a6d11bab12 WatchSource:0}: Error finding container f837153e2c2410be305fccfc5d0a0c9f41401bddfafba7aa24e0b4a6d11bab12: Status 404 returned error can't find the container with id f837153e2c2410be305fccfc5d0a0c9f41401bddfafba7aa24e0b4a6d11bab12
Sep 10 12:15:45.859391 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:45.859317    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"]
Sep 10 12:15:45.859976 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:15:45.859934    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fddc2a7_8f0d_49e5_bb63_66ba1565154a.slice/crio-2c78dd1471cb53fb27370267a241a76ec375d5e75e752bdf0f0500401e414262 WatchSource:0}: Error finding container 2c78dd1471cb53fb27370267a241a76ec375d5e75e752bdf0f0500401e414262: Status 404 returned error can't find the container with id 2c78dd1471cb53fb27370267a241a76ec375d5e75e752bdf0f0500401e414262
Sep 10 12:15:46.178108 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:46.178063    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp" event={"ID":"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f","Type":"ContainerStarted","Data":"f837153e2c2410be305fccfc5d0a0c9f41401bddfafba7aa24e0b4a6d11bab12"}
Sep 10 12:15:46.179762 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:46.179730    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq" event={"ID":"8fddc2a7-8f0d-49e5-bb63-66ba1565154a","Type":"ContainerStarted","Data":"06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc"}
Sep 10 12:15:46.179901 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:46.179785    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq" event={"ID":"8fddc2a7-8f0d-49e5-bb63-66ba1565154a","Type":"ContainerStarted","Data":"2c78dd1471cb53fb27370267a241a76ec375d5e75e752bdf0f0500401e414262"}
Sep 10 12:15:46.179901 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:46.179852    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:15:46.205125 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:46.205081    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq" podStartSLOduration=1.205066608 podStartE2EDuration="1.205066608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:46.202880727 +0000 UTC m=+1456.806155639" watchObservedRunningTime="2026-09-10 12:15:46.205066608 +0000 UTC m=+1456.808341520"
Sep 10 12:15:47.188032 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:47.187982    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp" event={"ID":"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f","Type":"ContainerStarted","Data":"4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082"}
Sep 10 12:15:47.241033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:47.240968    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp" podStartSLOduration=2.6064106049999998 podStartE2EDuration="3.240950257s" totalImagesPullingTime="634.539652ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:15:47.239956501 +0000 UTC m=+1457.843231412" watchObservedRunningTime="2026-09-10 12:15:47.240950257 +0000 UTC m=+1457.844225197"
Sep 10 12:15:48.193409 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:48.193379    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:52.665685 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:52.665644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:15:54.204276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:54.204242    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:15:57.190324 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:15:57.190293    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:16:05.507369 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:05.507336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:06.282942 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:06.282910    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:07.068112 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:07.068078    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:08.585045 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:08.585008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:11.701520 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:11.701488    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:26.960037 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:26.960005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:26.982095 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:26.982061    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"]
Sep 10 12:16:26.982354 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:26.982328    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f" podUID="598b85a4-200b-4aa5-9b6c-743dac6a09fd" containerName="payload-processing" containerID="cri-o://2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f" gracePeriod=30
Sep 10 12:16:26.993951 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:26.993916    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"]
Sep 10 12:16:26.994415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:26.994345    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6" podUID="0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" containerName="payload-pre-processing" containerID="cri-o://5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6" gracePeriod=30
Sep 10 12:16:27.265107 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.265065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:27.271463 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.271429    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:16:27.274200 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.274181    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:16:27.327176 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.326255    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7grn2\" (UniqueName: \"kubernetes.io/projected/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-kube-api-access-7grn2\") pod \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") "
Sep 10 12:16:27.327176 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.326543    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-plugins-config-volume\") pod \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\" (UID: \"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce\") "
Sep 10 12:16:27.327176 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.326689    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vz59\" (UniqueName: \"kubernetes.io/projected/598b85a4-200b-4aa5-9b6c-743dac6a09fd-kube-api-access-8vz59\") pod \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") "
Sep 10 12:16:27.327176 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.326765    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/598b85a4-200b-4aa5-9b6c-743dac6a09fd-plugins-config-volume\") pod \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\" (UID: \"598b85a4-200b-4aa5-9b6c-743dac6a09fd\") "
Sep 10 12:16:27.327176 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.326985    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" (UID: "0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:16:27.327623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.327213    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:27.327623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.327573    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598b85a4-200b-4aa5-9b6c-743dac6a09fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "598b85a4-200b-4aa5-9b6c-743dac6a09fd" (UID: "598b85a4-200b-4aa5-9b6c-743dac6a09fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:16:27.329055 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.329031    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-kube-api-access-7grn2" (OuterVolumeSpecName: "kube-api-access-7grn2") pod "0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" (UID: "0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce"). InnerVolumeSpecName "kube-api-access-7grn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:16:27.329518 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.329495    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598b85a4-200b-4aa5-9b6c-743dac6a09fd-kube-api-access-8vz59" (OuterVolumeSpecName: "kube-api-access-8vz59") pod "598b85a4-200b-4aa5-9b6c-743dac6a09fd" (UID: "598b85a4-200b-4aa5-9b6c-743dac6a09fd"). InnerVolumeSpecName "kube-api-access-8vz59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:16:27.337289 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.336112    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:27.420503 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.420467    2571 generic.go:356] "Generic (PLEG): container finished" podID="598b85a4-200b-4aa5-9b6c-743dac6a09fd" containerID="2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f" exitCode=0
Sep 10 12:16:27.420701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.420518    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"
Sep 10 12:16:27.420701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.420556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f" event={"ID":"598b85a4-200b-4aa5-9b6c-743dac6a09fd","Type":"ContainerDied","Data":"2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f"}
Sep 10 12:16:27.420701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.420615    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f" event={"ID":"598b85a4-200b-4aa5-9b6c-743dac6a09fd","Type":"ContainerDied","Data":"dba76ecb1fc249fb600dcf67fea6645cb7d28c68ed5f7e19abc2c5de9dbb05f4"}
Sep 10 12:16:27.420701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.420644    2571 scope.go:117] "RemoveContainer" containerID="2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f"
Sep 10 12:16:27.422010 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.421983    2571 generic.go:356] "Generic (PLEG): container finished" podID="0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" containerID="5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6" exitCode=0
Sep 10 12:16:27.422099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.422017    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6" event={"ID":"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce","Type":"ContainerDied","Data":"5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6"}
Sep 10 12:16:27.422099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.422041    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"
Sep 10 12:16:27.422099 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.422067    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6" event={"ID":"0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce","Type":"ContainerDied","Data":"7a83576ee68849f85bae56ed32c1f0f04617f90269890e703e012275c516ccc0"}
Sep 10 12:16:27.427847 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.427823    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7grn2\" (UniqueName: \"kubernetes.io/projected/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce-kube-api-access-7grn2\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:27.427847 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.427848    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vz59\" (UniqueName: \"kubernetes.io/projected/598b85a4-200b-4aa5-9b6c-743dac6a09fd-kube-api-access-8vz59\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:27.428033 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.427863    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/598b85a4-200b-4aa5-9b6c-743dac6a09fd-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:27.430169 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.430133    2571 scope.go:117] "RemoveContainer" containerID="2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f"
Sep 10 12:16:27.430599 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:16:27.430525    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f\": container with ID starting with 2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f not found: ID does not exist" containerID="2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f"
Sep 10 12:16:27.430599 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.430576    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f"} err="failed to get container status \"2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f\": rpc error: code = NotFound desc = could not find container \"2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f\": container with ID starting with 2d3516ade0ced32d231374c4fbfb9979441beb122c9f93a31e840f8bc456d40f not found: ID does not exist"
Sep 10 12:16:27.430726 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.430603    2571 scope.go:117] "RemoveContainer" containerID="5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6"
Sep 10 12:16:27.440816 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.440796    2571 scope.go:117] "RemoveContainer" containerID="5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6"
Sep 10 12:16:27.441141 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:16:27.441122    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6\": container with ID starting with 5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6 not found: ID does not exist" containerID="5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6"
Sep 10 12:16:27.441238 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.441162    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6"} err="failed to get container status \"5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6\": rpc error: code = NotFound desc = could not find container \"5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6\": container with ID starting with 5710689dab075dd35a5ec981f8193fdb6c9c38ec28a93e6f607815774e1c88a6 not found: ID does not exist"
Sep 10 12:16:27.447963 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.447941    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"]
Sep 10 12:16:27.453167 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.453136    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-187e7e-95bb75c5d-jcfs6"]
Sep 10 12:16:27.463949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.463922    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"]
Sep 10 12:16:27.467905 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.467881    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-187e7e-5f8bbdc7c-kmh9f"]
Sep 10 12:16:27.528595 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.528516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:27.893211 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.893124    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" path="/var/lib/kubelet/pods/0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce/volumes"
Sep 10 12:16:27.893539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:27.893525    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="598b85a4-200b-4aa5-9b6c-743dac6a09fd" path="/var/lib/kubelet/pods/598b85a4-200b-4aa5-9b6c-743dac6a09fd/volumes"
Sep 10 12:16:30.087146 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:30.087120    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"]
Sep 10 12:16:30.109891 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:30.109858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:31.105835 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:31.105794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:33.345765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:33.345732    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:33.459802 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:33.459764    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:33.669747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:33.669665    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:34.068402 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:34.068360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:36.697019 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:36.696988    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:39.217367 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:39.217336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:44.172434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:44.172391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:45.108573 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.108536    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28" podUID="0aedb7fc-5321-4772-9f8f-255575cc09c4" containerName="main" containerID="cri-o://0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5" gracePeriod=45
Sep 10 12:16:45.367663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.367607    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:16:45.491878 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.491845    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aedb7fc-5321-4772-9f8f-255575cc09c4-tls-certs\") pod \"0aedb7fc-5321-4772-9f8f-255575cc09c4\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") "
Sep 10 12:16:45.491878 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.491882    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-home\") pod \"0aedb7fc-5321-4772-9f8f-255575cc09c4\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") "
Sep 10 12:16:45.492137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.491910    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6rsm\" (UniqueName: \"kubernetes.io/projected/0aedb7fc-5321-4772-9f8f-255575cc09c4-kube-api-access-r6rsm\") pod \"0aedb7fc-5321-4772-9f8f-255575cc09c4\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") "
Sep 10 12:16:45.492137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.491956    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-tmp-dir\") pod \"0aedb7fc-5321-4772-9f8f-255575cc09c4\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") "
Sep 10 12:16:45.492137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.491983    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-model-cache\") pod \"0aedb7fc-5321-4772-9f8f-255575cc09c4\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") "
Sep 10 12:16:45.492137 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.492086    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-dshm\") pod \"0aedb7fc-5321-4772-9f8f-255575cc09c4\" (UID: \"0aedb7fc-5321-4772-9f8f-255575cc09c4\") "
Sep 10 12:16:45.492382 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.492203    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-home" (OuterVolumeSpecName: "home") pod "0aedb7fc-5321-4772-9f8f-255575cc09c4" (UID: "0aedb7fc-5321-4772-9f8f-255575cc09c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:16:45.492382 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.492245    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0aedb7fc-5321-4772-9f8f-255575cc09c4" (UID: "0aedb7fc-5321-4772-9f8f-255575cc09c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:16:45.492465 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.492447    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-home\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:45.492511 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.492475    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-tmp-dir\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:45.492511 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.492443    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "0aedb7fc-5321-4772-9f8f-255575cc09c4" (UID: "0aedb7fc-5321-4772-9f8f-255575cc09c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:16:45.494337 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.494314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aedb7fc-5321-4772-9f8f-255575cc09c4-kube-api-access-r6rsm" (OuterVolumeSpecName: "kube-api-access-r6rsm") pod "0aedb7fc-5321-4772-9f8f-255575cc09c4" (UID: "0aedb7fc-5321-4772-9f8f-255575cc09c4"). InnerVolumeSpecName "kube-api-access-r6rsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:16:45.494425 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.494310    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-dshm" (OuterVolumeSpecName: "dshm") pod "0aedb7fc-5321-4772-9f8f-255575cc09c4" (UID: "0aedb7fc-5321-4772-9f8f-255575cc09c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:16:45.494425 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.494375    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aedb7fc-5321-4772-9f8f-255575cc09c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0aedb7fc-5321-4772-9f8f-255575cc09c4" (UID: "0aedb7fc-5321-4772-9f8f-255575cc09c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:16:45.536358 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.536319    2571 generic.go:356] "Generic (PLEG): container finished" podID="0aedb7fc-5321-4772-9f8f-255575cc09c4" containerID="0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5" exitCode=0
Sep 10 12:16:45.536358 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.536359    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"
Sep 10 12:16:45.536597 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.536403    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28" event={"ID":"0aedb7fc-5321-4772-9f8f-255575cc09c4","Type":"ContainerDied","Data":"0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5"}
Sep 10 12:16:45.536597 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.536446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28" event={"ID":"0aedb7fc-5321-4772-9f8f-255575cc09c4","Type":"ContainerDied","Data":"1d757698f988b4ca40bfd31b99734c9d60444813712290869fe38abc7432a575"}
Sep 10 12:16:45.536597 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.536463    2571 scope.go:117] "RemoveContainer" containerID="0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5"
Sep 10 12:16:45.546441 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.546421    2571 scope.go:117] "RemoveContainer" containerID="0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5"
Sep 10 12:16:45.546721 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:16:45.546700    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5\": container with ID starting with 0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5 not found: ID does not exist" containerID="0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5"
Sep 10 12:16:45.546803 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.546730    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5"} err="failed to get container status \"0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5\": rpc error: code = NotFound desc = could not find container \"0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5\": container with ID starting with 0b90fcef34310584735a024730479c724fcbdce06d3e80910c9ac90dca0e71b5 not found: ID does not exist"
Sep 10 12:16:45.561163 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.561121    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"]
Sep 10 12:16:45.565056 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.565033    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-c07be0/sub-test-model-c07be0-kserve-5696d6665-9bv28"]
Sep 10 12:16:45.593403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.593365    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6rsm\" (UniqueName: \"kubernetes.io/projected/0aedb7fc-5321-4772-9f8f-255575cc09c4-kube-api-access-r6rsm\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:45.593403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.593400    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-model-cache\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:45.593403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.593411    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aedb7fc-5321-4772-9f8f-255575cc09c4-dshm\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:45.593589 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.593419    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aedb7fc-5321-4772-9f8f-255575cc09c4-tls-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:45.893203 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:45.893138    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aedb7fc-5321-4772-9f8f-255575cc09c4" path="/var/lib/kubelet/pods/0aedb7fc-5321-4772-9f8f-255575cc09c4/volumes"
Sep 10 12:16:49.754749 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.754719    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"]
Sep 10 12:16:49.755209 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.754922    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp" podUID="92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" containerName="maas-api" containerID="cri-o://4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082" gracePeriod=30
Sep 10 12:16:49.787218 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.787184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:49.829038 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.829000    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"]
Sep 10 12:16:49.829434 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.829399    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq" podUID="8fddc2a7-8f0d-49e5-bb63-66ba1565154a" containerName="payload-pre-processing" containerID="cri-o://06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc" gracePeriod=30
Sep 10 12:16:49.871884 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.871854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:49.972638 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:49.972606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:50.070829 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.070802    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:16:50.073324 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.073292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:50.126674 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.126650    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:16:50.132095 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132069    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-metrics-certs\") pod \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") "
Sep 10 12:16:50.132221 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132130    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-plugins-config-volume\") pod \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") "
Sep 10 12:16:50.132221 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132200    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-openshift-service-ca\") pod \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") "
Sep 10 12:16:50.132345 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132273    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l5zq\" (UniqueName: \"kubernetes.io/projected/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-kube-api-access-6l5zq\") pod \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") "
Sep 10 12:16:50.132345 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132329    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf4w2\" (UniqueName: \"kubernetes.io/projected/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-kube-api-access-cf4w2\") pod \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\" (UID: \"8fddc2a7-8f0d-49e5-bb63-66ba1565154a\") "
Sep 10 12:16:50.132453 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132364    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls\") pod \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\" (UID: \"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f\") "
Sep 10 12:16:50.132534 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132499    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8fddc2a7-8f0d-49e5-bb63-66ba1565154a" (UID: "8fddc2a7-8f0d-49e5-bb63-66ba1565154a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:16:50.132590 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132543    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" (UID: "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:16:50.132731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132706    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:50.132731 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.132728    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:50.134375 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.134341    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" (UID: "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:16:50.134507 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.134408    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" (UID: "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:16:50.134800 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.134776    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-kube-api-access-cf4w2" (OuterVolumeSpecName: "kube-api-access-cf4w2") pod "8fddc2a7-8f0d-49e5-bb63-66ba1565154a" (UID: "8fddc2a7-8f0d-49e5-bb63-66ba1565154a"). InnerVolumeSpecName "kube-api-access-cf4w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:16:50.134875 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.134813    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-kube-api-access-6l5zq" (OuterVolumeSpecName: "kube-api-access-6l5zq") pod "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" (UID: "92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f"). InnerVolumeSpecName "kube-api-access-6l5zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:16:50.233315 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.233284    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:50.233315 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.233309    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l5zq\" (UniqueName: \"kubernetes.io/projected/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-kube-api-access-6l5zq\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:50.233315 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.233322    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf4w2\" (UniqueName: \"kubernetes.io/projected/8fddc2a7-8f0d-49e5-bb63-66ba1565154a-kube-api-access-cf4w2\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:50.233554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.233330    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:16:50.493117 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.493082    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:50.566311 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.566278    2571 generic.go:356] "Generic (PLEG): container finished" podID="8fddc2a7-8f0d-49e5-bb63-66ba1565154a" containerID="06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc" exitCode=0
Sep 10 12:16:50.566485 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.566326    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"
Sep 10 12:16:50.566485 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.566375    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq" event={"ID":"8fddc2a7-8f0d-49e5-bb63-66ba1565154a","Type":"ContainerDied","Data":"06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc"}
Sep 10 12:16:50.566485 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.566430    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq" event={"ID":"8fddc2a7-8f0d-49e5-bb63-66ba1565154a","Type":"ContainerDied","Data":"2c78dd1471cb53fb27370267a241a76ec375d5e75e752bdf0f0500401e414262"}
Sep 10 12:16:50.566485 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.566453    2571 scope.go:117] "RemoveContainer" containerID="06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc"
Sep 10 12:16:50.568043 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.568021    2571 generic.go:356] "Generic (PLEG): container finished" podID="92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" containerID="4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082" exitCode=0
Sep 10 12:16:50.568184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.568053    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp" event={"ID":"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f","Type":"ContainerDied","Data":"4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082"}
Sep 10 12:16:50.568184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.568065    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"
Sep 10 12:16:50.568184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.568090    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp" event={"ID":"92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f","Type":"ContainerDied","Data":"f837153e2c2410be305fccfc5d0a0c9f41401bddfafba7aa24e0b4a6d11bab12"}
Sep 10 12:16:50.575504 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.575486    2571 scope.go:117] "RemoveContainer" containerID="06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc"
Sep 10 12:16:50.575828 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:16:50.575803    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc\": container with ID starting with 06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc not found: ID does not exist" containerID="06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc"
Sep 10 12:16:50.575923 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.575839    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc"} err="failed to get container status \"06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc\": rpc error: code = NotFound desc = could not find container \"06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc\": container with ID starting with 06c169e38a686e7dc01625da14673ff6466cf6c2dc0e3dc582cb836804667bbc not found: ID does not exist"
Sep 10 12:16:50.575923 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.575863    2571 scope.go:117] "RemoveContainer" containerID="4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082"
Sep 10 12:16:50.584826 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.584810    2571 scope.go:117] "RemoveContainer" containerID="4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082"
Sep 10 12:16:50.585080 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:16:50.585061    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082\": container with ID starting with 4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082 not found: ID does not exist" containerID="4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082"
Sep 10 12:16:50.585145 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.585086    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082"} err="failed to get container status \"4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082\": rpc error: code = NotFound desc = could not find container \"4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082\": container with ID starting with 4ff9b3a385f903dae39c94efb197c49806d33c1465c1c919b33e7f237f8d6082 not found: ID does not exist"
Sep 10 12:16:50.616404 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.616380    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"]
Sep 10 12:16:50.628412 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.628387    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-c531cb-59f5957c46-pmcwq"]
Sep 10 12:16:50.650721 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.650696    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"]
Sep 10 12:16:50.655681 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:50.655662    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c531cb-7b64598947-k46lp"]
Sep 10 12:16:51.892774 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:51.892741    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fddc2a7-8f0d-49e5-bb63-66ba1565154a" path="/var/lib/kubelet/pods/8fddc2a7-8f0d-49e5-bb63-66ba1565154a/volumes"
Sep 10 12:16:51.893148 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:51.893135    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" path="/var/lib/kubelet/pods/92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f/volumes"
Sep 10 12:16:53.914426 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:53.914393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:56.707481 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:56.707444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:56.798341 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:56.798306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:56.999889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:56.999809    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:58.598144 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:58.598102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:59.159965 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:59.159930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:16:59.289602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:16:59.289570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:08.038952 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.038917    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"]
Sep 10 12:17:08.039504 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.039198    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm" podUID="f3553857-7450-49b4-b247-5f1601862ae4" containerName="maas-api" containerID="cri-o://71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9" gracePeriod=30
Sep 10 12:17:08.071961 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.071923    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"]
Sep 10 12:17:08.080581 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.077313    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-639beb-29817375-lh8fl"]
Sep 10 12:17:08.116603 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.116561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:08.216750 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.216716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:08.289744 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.289718    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:17:08.296260 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.296210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:08.391829 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.391788    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:08.403011 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.402973    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3553857-7450-49b4-b247-5f1601862ae4-openshift-service-ca\") pod \"f3553857-7450-49b4-b247-5f1601862ae4\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") "
Sep 10 12:17:08.403198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.403032    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26xbs\" (UniqueName: \"kubernetes.io/projected/f3553857-7450-49b4-b247-5f1601862ae4-kube-api-access-26xbs\") pod \"f3553857-7450-49b4-b247-5f1601862ae4\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") "
Sep 10 12:17:08.403198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.403094    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-metrics-certs\") pod \"f3553857-7450-49b4-b247-5f1601862ae4\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") "
Sep 10 12:17:08.403198 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.403178    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-maas-api-tls\") pod \"f3553857-7450-49b4-b247-5f1601862ae4\" (UID: \"f3553857-7450-49b4-b247-5f1601862ae4\") "
Sep 10 12:17:08.403388 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.403367    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3553857-7450-49b4-b247-5f1601862ae4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3553857-7450-49b4-b247-5f1601862ae4" (UID: "f3553857-7450-49b4-b247-5f1601862ae4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:17:08.403547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.403523    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3553857-7450-49b4-b247-5f1601862ae4-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:08.405343 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.405314    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f3553857-7450-49b4-b247-5f1601862ae4" (UID: "f3553857-7450-49b4-b247-5f1601862ae4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:17:08.405455 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.405382    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3553857-7450-49b4-b247-5f1601862ae4-kube-api-access-26xbs" (OuterVolumeSpecName: "kube-api-access-26xbs") pod "f3553857-7450-49b4-b247-5f1601862ae4" (UID: "f3553857-7450-49b4-b247-5f1601862ae4"). InnerVolumeSpecName "kube-api-access-26xbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:08.405455 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.405429    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3553857-7450-49b4-b247-5f1601862ae4" (UID: "f3553857-7450-49b4-b247-5f1601862ae4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:17:08.504237 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.504130    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:08.504237 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.504179    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3553857-7450-49b4-b247-5f1601862ae4-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:08.504237 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.504193    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26xbs\" (UniqueName: \"kubernetes.io/projected/f3553857-7450-49b4-b247-5f1601862ae4-kube-api-access-26xbs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:08.666087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.666054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:08.671614 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.671587    2571 generic.go:356] "Generic (PLEG): container finished" podID="f3553857-7450-49b4-b247-5f1601862ae4" containerID="71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9" exitCode=0
Sep 10 12:17:08.671737 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.671632    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"
Sep 10 12:17:08.671737 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.671667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm" event={"ID":"f3553857-7450-49b4-b247-5f1601862ae4","Type":"ContainerDied","Data":"71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9"}
Sep 10 12:17:08.671823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.671749    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm" event={"ID":"f3553857-7450-49b4-b247-5f1601862ae4","Type":"ContainerDied","Data":"5d3c1bb0bdef82eec871127ab6947933380ad8ff3826001814fbb39c4769ef09"}
Sep 10 12:17:08.671823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.671772    2571 scope.go:117] "RemoveContainer" containerID="71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9"
Sep 10 12:17:08.683532 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.683328    2571 scope.go:117] "RemoveContainer" containerID="71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9"
Sep 10 12:17:08.684031 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:17:08.684012    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9\": container with ID starting with 71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9 not found: ID does not exist" containerID="71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9"
Sep 10 12:17:08.684115 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.684046    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9"} err="failed to get container status \"71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9\": rpc error: code = NotFound desc = could not find container \"71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9\": container with ID starting with 71fe2d3ac8ac9366de7a028abfb72ed1aa43c5fae2d21977b8d7d01478b0e6a9 not found: ID does not exist"
Sep 10 12:17:08.697485 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.697460    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"]
Sep 10 12:17:08.704127 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:08.704098    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-639beb-56dfb8b94f-dqgwm"]
Sep 10 12:17:09.891855 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:09.891822    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="948fafd0-3252-46d1-8466-5aea42bc4dc6" path="/var/lib/kubelet/pods/948fafd0-3252-46d1-8466-5aea42bc4dc6/volumes"
Sep 10 12:17:09.892276 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:09.892217    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3553857-7450-49b4-b247-5f1601862ae4" path="/var/lib/kubelet/pods/f3553857-7450-49b4-b247-5f1601862ae4/volumes"
Sep 10 12:17:14.738656 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:14.738626    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:14.813955 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:14.813861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:15.018215 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:15.018101    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"]
Sep 10 12:17:15.018394 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:15.018333    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy" containerID="cri-o://66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4" gracePeriod=30
Sep 10 12:17:15.055533 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:15.055505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:15.494938 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:15.494903    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:17.459571 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:17.459534    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:17.616990 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:17.616958    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:18.632461 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:18.632429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:20.262722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.262692    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:17:20.410480 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410381    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-socket\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410480 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410431    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-data\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410480 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410449    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-certs\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410487    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-token\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410519    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-podinfo\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410554    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-credential-socket\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410614    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-envoy\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410639    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42117b39-84c6-4e4f-b99f-f1a934a8608c-istiod-ca-cert\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410683    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfzsb\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-kube-api-access-mfzsb\") pod \"42117b39-84c6-4e4f-b99f-f1a934a8608c\" (UID: \"42117b39-84c6-4e4f-b99f-f1a934a8608c\") "
Sep 10 12:17:20.410776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410722    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:20.411143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410779    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-data" (OuterVolumeSpecName: "istio-data") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:20.411143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.410937    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:20.411143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.411092    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.411143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.411113    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-data\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.411143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.411128    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-workload-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.411143 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.411137    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42117b39-84c6-4e4f-b99f-f1a934a8608c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:17:20.411500 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.411225    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:20.412838 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.412807    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 12:17:20.412943 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.412929    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-kube-api-access-mfzsb" (OuterVolumeSpecName: "kube-api-access-mfzsb") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "kube-api-access-mfzsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:20.412943 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.412932    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-token" (OuterVolumeSpecName: "istio-token") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:20.413028 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.412941    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "42117b39-84c6-4e4f-b99f-f1a934a8608c" (UID: "42117b39-84c6-4e4f-b99f-f1a934a8608c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:17:20.512664 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.512622    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfzsb\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-kube-api-access-mfzsb\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.512664 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.512656    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-token\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.512664 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.512665    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-podinfo\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.512664 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.512673    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-credential-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.512945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.512681    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42117b39-84c6-4e4f-b99f-f1a934a8608c-istio-envoy\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.512945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.512689    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42117b39-84c6-4e4f-b99f-f1a934a8608c-istiod-ca-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:20.738672 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.738636    2571 generic.go:356] "Generic (PLEG): container finished" podID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerID="66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4" exitCode=0
Sep 10 12:17:20.738822 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.738705    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"
Sep 10 12:17:20.738822 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.738710    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" event={"ID":"42117b39-84c6-4e4f-b99f-f1a934a8608c","Type":"ContainerDied","Data":"66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4"}
Sep 10 12:17:20.738822 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.738794    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6" event={"ID":"42117b39-84c6-4e4f-b99f-f1a934a8608c","Type":"ContainerDied","Data":"011009510ebf8e65aaaa5fc1a177003f3a7ccc0c0b02683e3aff9a2cbf4104a9"}
Sep 10 12:17:20.738954 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.738824    2571 scope.go:117] "RemoveContainer" containerID="66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4"
Sep 10 12:17:20.748065 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.748043    2571 scope.go:117] "RemoveContainer" containerID="36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"
Sep 10 12:17:20.756269 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.756243    2571 scope.go:117] "RemoveContainer" containerID="66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4"
Sep 10 12:17:20.756553 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:17:20.756533    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4\": container with ID starting with 66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4 not found: ID does not exist" containerID="66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4"
Sep 10 12:17:20.756604 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.756563    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4"} err="failed to get container status \"66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4\": rpc error: code = NotFound desc = could not find container \"66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4\": container with ID starting with 66da954318d9fafb0bfad855f1e7b14a415e67eb982af70ac63af8ac6c995dd4 not found: ID does not exist"
Sep 10 12:17:20.756604 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.756582    2571 scope.go:117] "RemoveContainer" containerID="36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"
Sep 10 12:17:20.756848 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:17:20.756829    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240\": container with ID starting with 36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240 not found: ID does not exist" containerID="36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"
Sep 10 12:17:20.756890 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.756855    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240"} err="failed to get container status \"36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240\": rpc error: code = NotFound desc = could not find container \"36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240\": container with ID starting with 36f710050beb9a7dcd0556779ee4fad476a56f150de61055e61921ee34573240 not found: ID does not exist"
Sep 10 12:17:20.776722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.776690    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"]
Sep 10 12:17:20.789548 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:20.789517    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-639beb-openshift-default-db468dd6d-bd7l6"]
Sep 10 12:17:21.893139 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:21.893101    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" path="/var/lib/kubelet/pods/42117b39-84c6-4e4f-b99f-f1a934a8608c/volumes"
Sep 10 12:17:26.040427 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.040398    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:26.109732 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.109694    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"]
Sep 10 12:17:26.110031 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.109982    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s" podUID="5489a227-2ef3-4332-8b64-66a3de100725" containerName="payload-pre-processing" containerID="cri-o://5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c" gracePeriod=30
Sep 10 12:17:26.166712 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.166690    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:26.296706 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.296630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:26.361753 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.361730    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:17:26.464477 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.464444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:26.468419 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.468398    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5489a227-2ef3-4332-8b64-66a3de100725-plugins-config-volume\") pod \"5489a227-2ef3-4332-8b64-66a3de100725\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") "
Sep 10 12:17:26.468544 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.468459    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zp49\" (UniqueName: \"kubernetes.io/projected/5489a227-2ef3-4332-8b64-66a3de100725-kube-api-access-7zp49\") pod \"5489a227-2ef3-4332-8b64-66a3de100725\" (UID: \"5489a227-2ef3-4332-8b64-66a3de100725\") "
Sep 10 12:17:26.468797 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.468770    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5489a227-2ef3-4332-8b64-66a3de100725-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5489a227-2ef3-4332-8b64-66a3de100725" (UID: "5489a227-2ef3-4332-8b64-66a3de100725"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:17:26.470500 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.470480    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5489a227-2ef3-4332-8b64-66a3de100725-kube-api-access-7zp49" (OuterVolumeSpecName: "kube-api-access-7zp49") pod "5489a227-2ef3-4332-8b64-66a3de100725" (UID: "5489a227-2ef3-4332-8b64-66a3de100725"). InnerVolumeSpecName "kube-api-access-7zp49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:17:26.569532 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.569456    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5489a227-2ef3-4332-8b64-66a3de100725-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:26.569532 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.569481    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zp49\" (UniqueName: \"kubernetes.io/projected/5489a227-2ef3-4332-8b64-66a3de100725-kube-api-access-7zp49\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:17:26.601815 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.601780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:26.772818 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.772783    2571 generic.go:356] "Generic (PLEG): container finished" podID="5489a227-2ef3-4332-8b64-66a3de100725" containerID="5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c" exitCode=0
Sep 10 12:17:26.772988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.772829    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"
Sep 10 12:17:26.772988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.772867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s" event={"ID":"5489a227-2ef3-4332-8b64-66a3de100725","Type":"ContainerDied","Data":"5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c"}
Sep 10 12:17:26.772988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.772924    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s" event={"ID":"5489a227-2ef3-4332-8b64-66a3de100725","Type":"ContainerDied","Data":"3ce9ba445dc6250ad42d2158292b72c1f6aeb203c461031457a2a553006ffe17"}
Sep 10 12:17:26.772988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.772952    2571 scope.go:117] "RemoveContainer" containerID="5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c"
Sep 10 12:17:26.781921 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.781895    2571 scope.go:117] "RemoveContainer" containerID="5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c"
Sep 10 12:17:26.782229 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:17:26.782211    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c\": container with ID starting with 5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c not found: ID does not exist" containerID="5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c"
Sep 10 12:17:26.782303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.782236    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c"} err="failed to get container status \"5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c\": rpc error: code = NotFound desc = could not find container \"5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c\": container with ID starting with 5628db7e701e7ebe4cd05d91860b690cccb1c3b92c10942ac25366aa505a993c not found: ID does not exist"
Sep 10 12:17:26.825013 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.824951    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"]
Sep 10 12:17:26.835648 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.835624    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c66562-f9bf4dcc8-qrz6s"]
Sep 10 12:17:26.954429 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:26.954399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:27.892169 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:27.892129    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5489a227-2ef3-4332-8b64-66a3de100725" path="/var/lib/kubelet/pods/5489a227-2ef3-4332-8b64-66a3de100725/volumes"
Sep 10 12:17:32.978316 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:32.978273    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:33.077650 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:33.077605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:17:33.249945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:17:33.249854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:23.484813 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:23.484694    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:25.432459 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.432358    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"]
Sep 10 12:18:25.433185 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433147    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5489a227-2ef3-4332-8b64-66a3de100725" containerName="payload-pre-processing"
Sep 10 12:18:25.433185 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433185    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5489a227-2ef3-4332-8b64-66a3de100725" containerName="payload-pre-processing"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433206    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aedb7fc-5321-4772-9f8f-255575cc09c4" containerName="main"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433216    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aedb7fc-5321-4772-9f8f-255575cc09c4" containerName="main"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433226    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" containerName="payload-pre-processing"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433234    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" containerName="payload-pre-processing"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433257    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433265    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433276    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" containerName="maas-api"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433284    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" containerName="maas-api"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433306    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3553857-7450-49b4-b247-5f1601862ae4" containerName="maas-api"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433314    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3553857-7450-49b4-b247-5f1601862ae4" containerName="maas-api"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433330    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433338    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433353    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598b85a4-200b-4aa5-9b6c-743dac6a09fd" containerName="payload-processing"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433362    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="598b85a4-200b-4aa5-9b6c-743dac6a09fd" containerName="payload-processing"
Sep 10 12:18:25.433415 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433418    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fddc2a7-8f0d-49e5-bb63-66ba1565154a" containerName="payload-pre-processing"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433429    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fddc2a7-8f0d-49e5-bb63-66ba1565154a" containerName="payload-pre-processing"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433649    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="92efd4e4-c1f6-4dc4-a0fc-ad5ab36b8d8f" containerName="maas-api"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433670    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3553857-7450-49b4-b247-5f1601862ae4" containerName="maas-api"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433686    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aedb7fc-5321-4772-9f8f-255575cc09c4" containerName="main"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433695    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433705    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0adaa6ef-2f81-479e-9bb4-e8e6c8dd5cce" containerName="payload-pre-processing"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433715    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="42117b39-84c6-4e4f-b99f-f1a934a8608c" containerName="istio-proxy"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433729    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5489a227-2ef3-4332-8b64-66a3de100725" containerName="payload-pre-processing"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433738    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="598b85a4-200b-4aa5-9b6c-743dac6a09fd" containerName="payload-processing"
Sep 10 12:18:25.434303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.433747    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fddc2a7-8f0d-49e5-bb63-66ba1565154a" containerName="payload-pre-processing"
Sep 10 12:18:25.439232 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.439208    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.441573 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.441552    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-19f6c3-openshift-default-dockercfg-qdhbg\""
Sep 10 12:18:25.449535 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.449510    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"]
Sep 10 12:18:25.567370 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6006247-8ab1-4956-9450-0fac48291ce2-istio-podinfo\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567381    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-istio-token\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567415    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-socket\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567452    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-data\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567468    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55vx\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-kube-api-access-s55vx\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567495    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-credential-socket\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567547 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-certs\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567769 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6006247-8ab1-4956-9450-0fac48291ce2-istiod-ca-cert\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.567769 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.567604    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-envoy\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668570 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668526    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6006247-8ab1-4956-9450-0fac48291ce2-istiod-ca-cert\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668570 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-envoy\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668605    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6006247-8ab1-4956-9450-0fac48291ce2-istio-podinfo\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668646    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-istio-token\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668672    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-socket\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668727    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-data\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668754    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55vx\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-kube-api-access-s55vx\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668785    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-credential-socket\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.668854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.668817    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-certs\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.669308 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.669147    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-data\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.669308 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.669206    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-certs\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.669423 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.669390    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-socket\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.669486 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.669460    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-credential-socket\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.669548 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.669483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6006247-8ab1-4956-9450-0fac48291ce2-istiod-ca-cert\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.671207 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.671184    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-envoy\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.671654 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.671637    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6006247-8ab1-4956-9450-0fac48291ce2-istio-podinfo\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.679749 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.679724    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55vx\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-kube-api-access-s55vx\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.680818 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.680794    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-istio-token\") pod \"e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") " pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:25.750047 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:25.749950    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:26.034527 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:26.034503    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"]
Sep 10 12:18:26.036290 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:18:26.036264    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6006247_8ab1_4956_9450_0fac48291ce2.slice/crio-e0109795a966c00481879ef87af4ef4592d4db3dee22b69f75f7baa2d0454253 WatchSource:0}: Error finding container e0109795a966c00481879ef87af4ef4592d4db3dee22b69f75f7baa2d0454253: Status 404 returned error can't find the container with id e0109795a966c00481879ef87af4ef4592d4db3dee22b69f75f7baa2d0454253
Sep 10 12:18:26.038503 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:26.038471    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:18:26.038624 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:26.038534    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:18:26.038624 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:26.038566    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:18:26.080184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:26.080145    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" event={"ID":"b6006247-8ab1-4956-9450-0fac48291ce2","Type":"ContainerStarted","Data":"e0109795a966c00481879ef87af4ef4592d4db3dee22b69f75f7baa2d0454253"}
Sep 10 12:18:27.087139 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:27.087103    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" event={"ID":"b6006247-8ab1-4956-9450-0fac48291ce2","Type":"ContainerStarted","Data":"461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"}
Sep 10 12:18:27.109979 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:27.109930    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podStartSLOduration=2.109915825 podStartE2EDuration="2.109915825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:27.106957181 +0000 UTC m=+1617.710232106" watchObservedRunningTime="2026-09-10 12:18:27.109915825 +0000 UTC m=+1617.713190736"
Sep 10 12:18:27.319576 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:27.319537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:27.750744 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:27.750706    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:27.756316 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:27.756279    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:28.092367 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:28.092274    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:28.093306 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:28.093286    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:29.108863 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:29.108831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:29.204977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:29.204939    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:30.280636 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.280596    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"]
Sep 10 12:18:30.284359 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.284303    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.287499 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.287477    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-19f6c3-serving-cert\""
Sep 10 12:18:30.298928 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.298902    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"]
Sep 10 12:18:30.329043 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.329017    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:30.412697 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.412652    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-metrics-certs\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.412889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.412723    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98sqs\" (UniqueName: \"kubernetes.io/projected/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-kube-api-access-98sqs\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.412889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.412784    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.412889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.412871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.430423 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.430373    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:30.513722 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.513685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.513923 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.513732    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-metrics-certs\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.513923 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:18:30.513855    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-serving-cert: secret "maas-api-e2e-ipp-a-19f6c3-serving-cert" not found
Sep 10 12:18:30.514055 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.513857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98sqs\" (UniqueName: \"kubernetes.io/projected/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-kube-api-access-98sqs\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.514055 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:18:30.513929    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls podName:4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac nodeName:}" failed. No retries permitted until 2026-09-10 12:18:31.013905076 +0000 UTC m=+1621.617179971 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls") pod "maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" (UID: "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac") : secret "maas-api-e2e-ipp-a-19f6c3-serving-cert" not found
Sep 10 12:18:30.514055 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.514030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.514657 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.514639    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.516383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.516357    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-metrics-certs\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.522509 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.522487    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98sqs\" (UniqueName: \"kubernetes.io/projected/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-kube-api-access-98sqs\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:30.665296 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.665213    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:30.918940 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:30.918865    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:31.006757 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.006727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:31.019563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.019529    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:31.021926 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.021908    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls\") pod \"maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:31.057206 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.057180    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"]
Sep 10 12:18:31.060509 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.060493    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.062747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.062729    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-19f6c3\""
Sep 10 12:18:31.062844 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.062807    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-19f6c3-dockercfg-c58nj\""
Sep 10 12:18:31.080477 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.080451    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"]
Sep 10 12:18:31.196232 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.196199    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:31.221447 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.221414    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlwgf\" (UniqueName: \"kubernetes.io/projected/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-kube-api-access-mlwgf\") pod \"payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.221601 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.221549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.322184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.322130    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlwgf\" (UniqueName: \"kubernetes.io/projected/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-kube-api-access-mlwgf\") pod \"payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.322605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.322233    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.322897 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.322879    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.330997 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.330970    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlwgf\" (UniqueName: \"kubernetes.io/projected/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-kube-api-access-mlwgf\") pod \"payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.370716 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.370690    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:31.469717 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.469660    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"]
Sep 10 12:18:31.470833 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:18:31.470805    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0a5107_b8e7_45a7_bbdf_3259f2e9f6ac.slice/crio-14a5263dd6f1549bf4e7b9f3db09eb998f7f6b8d404149e60c26ff908344758c WatchSource:0}: Error finding container 14a5263dd6f1549bf4e7b9f3db09eb998f7f6b8d404149e60c26ff908344758c: Status 404 returned error can't find the container with id 14a5263dd6f1549bf4e7b9f3db09eb998f7f6b8d404149e60c26ff908344758c
Sep 10 12:18:31.642873 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:31.642816    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"]
Sep 10 12:18:31.645020 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:18:31.644991    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e19f8a_bccc_4a55_bb5c_c9569b8a5d7a.slice/crio-7d9bf7a95a516a29b584a03f8a714d5ae078e1f3d534bc7c92a4c8999f0768ae WatchSource:0}: Error finding container 7d9bf7a95a516a29b584a03f8a714d5ae078e1f3d534bc7c92a4c8999f0768ae: Status 404 returned error can't find the container with id 7d9bf7a95a516a29b584a03f8a714d5ae078e1f3d534bc7c92a4c8999f0768ae
Sep 10 12:18:32.114740 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.114694    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" event={"ID":"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac","Type":"ContainerStarted","Data":"d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38"}
Sep 10 12:18:32.114953 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.114815    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:32.114953 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.114844    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" event={"ID":"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac","Type":"ContainerStarted","Data":"14a5263dd6f1549bf4e7b9f3db09eb998f7f6b8d404149e60c26ff908344758c"}
Sep 10 12:18:32.116311 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.116288    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4" event={"ID":"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a","Type":"ContainerStarted","Data":"7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75"}
Sep 10 12:18:32.116439 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.116323    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4" event={"ID":"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a","Type":"ContainerStarted","Data":"7d9bf7a95a516a29b584a03f8a714d5ae078e1f3d534bc7c92a4c8999f0768ae"}
Sep 10 12:18:32.116439 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.116363    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:32.132884 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.132837    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" podStartSLOduration=1.6812080379999998 podStartE2EDuration="2.132822704s" totalImagesPullingTime="451.614666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:32.131495675 +0000 UTC m=+1622.734770586" watchObservedRunningTime="2026-09-10 12:18:32.132822704 +0000 UTC m=+1622.736097614"
Sep 10 12:18:32.151823 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:32.151778    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4" podStartSLOduration=1.151763137 podStartE2EDuration="1.151763137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:18:32.149463507 +0000 UTC m=+1622.752738412" watchObservedRunningTime="2026-09-10 12:18:32.151763137 +0000 UTC m=+1622.755038047"
Sep 10 12:18:38.127687 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:38.127658    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:18:43.123769 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:43.123741    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:18:44.134403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:44.134356    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 12:18:44.134843 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:44.134433    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 12:18:46.195225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:46.195190    2571 generic.go:356] "Generic (PLEG): container finished" podID="b6006247-8ab1-4956-9450-0fac48291ce2" containerID="461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a" exitCode=0
Sep 10 12:18:46.195621 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:46.195257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" event={"ID":"b6006247-8ab1-4956-9450-0fac48291ce2","Type":"ContainerDied","Data":"461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"}
Sep 10 12:18:46.195668 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:46.195643    2571 scope.go:117] "RemoveContainer" containerID="461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"
Sep 10 12:18:46.196695 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:46.196672    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:18:46.196750 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:46.196735    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:18:46.196785 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:46.196769    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 12:18:47.205309 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:47.205277    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" event={"ID":"b6006247-8ab1-4956-9450-0fac48291ce2","Type":"ContainerStarted","Data":"0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375"}
Sep 10 12:18:47.750374 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:47.750345    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:47.750580 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:47.750490    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 10 12:18:47.750580 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:47.750562    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 10 12:18:48.750976 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:48.750941    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 10 12:18:48.751427 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:48.750998    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 10 12:18:49.751059 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:49.751023    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 10 12:18:49.751465 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:49.751079    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 10 12:18:50.751356 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:50.751315    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 10 12:18:50.751776 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:50.751371    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 10 12:18:51.750741 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:51.750698    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 10 12:18:51.750924 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:51.750757    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 10 12:18:52.754921 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:52.754879    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:52.755400 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:52.755186    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:52.756060 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:52.756043    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:18:54.057828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:54.057794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:54.758141 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:54.758109    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:55.072100 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:55.072021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:55.578618 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:55.578585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:56.726504 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:56.726465    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:58.588121 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:58.588075    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:58.687008 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:58.686974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:59.782087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:59.782052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:18:59.862205 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:18:59.862171    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:19:00.199575 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:19:00.199543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:19:00.305988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:19:00.305947    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:19:00.400736 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:19:00.400699    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:19:23.621412 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:19:23.621377    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:19:28.769993 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:19:28.769955    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scdzb_b7713abc-e4fa-4b21-8453-a6fd6ad7a824/payload-processing/0.log"
Sep 10 12:19:37.093269 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:19:37.093239    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:03.771353 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:03.771305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:09.703303 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:09.703279    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4_10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a/payload-processing/0.log"
Sep 10 12:20:09.833450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:09.833423    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scdzb_b7713abc-e4fa-4b21-8453-a6fd6ad7a824/payload-processing/0.log"
Sep 10 12:20:10.229008 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:10.228974    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:10.571475 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:10.571396    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:10.969686 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:10.969656    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:13.998321 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:13.998285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:14.098842 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:14.098811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:15.243778 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.243747    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"]
Sep 10 12:20:15.247378 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.247358    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.250389 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.250357    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-2fc5d5-serving-cert\""
Sep 10 12:20:15.257055 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.257031    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"]
Sep 10 12:20:15.275649 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.275615    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:15.369313 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.369278    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:15.388181 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.388131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.388312 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.388180    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.388312 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.388265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.388403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.388334    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dthg8\" (UniqueName: \"kubernetes.io/projected/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-kube-api-access-dthg8\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.469526 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.469498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:15.489443 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.489417    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.489564 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.489494    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dthg8\" (UniqueName: \"kubernetes.io/projected/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-kube-api-access-dthg8\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.489564 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.489553    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.489644 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.489581    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.490242 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.490214    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.492063 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.492042    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.492183 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.492092    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.497847 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.497803    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dthg8\" (UniqueName: \"kubernetes.io/projected/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-kube-api-access-dthg8\") pod \"maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.559719 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.559679    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:15.772454 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.772342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:15.884041 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.884015    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"]
Sep 10 12:20:15.886246 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:20:15.886214    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb2d558_7949_4aa3_a6d3_4b8cd68242bc.slice/crio-9d86d6dab9fb73d6a41f529f320dc9b306a0a233db3a364fabab6616a51a86da WatchSource:0}: Error finding container 9d86d6dab9fb73d6a41f529f320dc9b306a0a233db3a364fabab6616a51a86da: Status 404 returned error can't find the container with id 9d86d6dab9fb73d6a41f529f320dc9b306a0a233db3a364fabab6616a51a86da
Sep 10 12:20:15.888009 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.887983    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:20:15.921597 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:15.921572    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:16.670880 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:16.670850    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl" event={"ID":"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc","Type":"ContainerStarted","Data":"9d86d6dab9fb73d6a41f529f320dc9b306a0a233db3a364fabab6616a51a86da"}
Sep 10 12:20:17.677462 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:17.677430    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl" event={"ID":"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc","Type":"ContainerStarted","Data":"ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d"}
Sep 10 12:20:17.677834 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:17.677551    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:17.702899 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:17.702850    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl" podStartSLOduration=1.947437571 podStartE2EDuration="2.702833598s" totalImagesPullingTime="755.396027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:20:17.701480978 +0000 UTC m=+1728.304755914" watchObservedRunningTime="2026-09-10 12:20:17.702833598 +0000 UTC m=+1728.306108543"
Sep 10 12:20:23.688069 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:23.688034    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:25.345668 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:25.345625    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:36.229291 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:36.229252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:39.017930 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:39.017895    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:45.551062 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.551019    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"]
Sep 10 12:20:45.551500 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.551349    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl" podUID="ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" containerName="maas-api" containerID="cri-o://ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d" gracePeriod=30
Sep 10 12:20:45.584957 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.584930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:45.686219 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.686186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:45.779016 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.778987    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:45.791772 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.791751    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:45.830243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.830163    2571 generic.go:356] "Generic (PLEG): container finished" podID="ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" containerID="ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d" exitCode=0
Sep 10 12:20:45.830243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.830206    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl" event={"ID":"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc","Type":"ContainerDied","Data":"ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d"}
Sep 10 12:20:45.830243 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.830221    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"
Sep 10 12:20:45.830469 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.830257    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl" event={"ID":"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc","Type":"ContainerDied","Data":"9d86d6dab9fb73d6a41f529f320dc9b306a0a233db3a364fabab6616a51a86da"}
Sep 10 12:20:45.830469 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.830285    2571 scope.go:117] "RemoveContainer" containerID="ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d"
Sep 10 12:20:45.839627 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.839605    2571 scope.go:117] "RemoveContainer" containerID="ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d"
Sep 10 12:20:45.839903 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:20:45.839883    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d\": container with ID starting with ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d not found: ID does not exist" containerID="ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d"
Sep 10 12:20:45.839980 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.839915    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d"} err="failed to get container status \"ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d\": rpc error: code = NotFound desc = could not find container \"ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d\": container with ID starting with ffe0892d5955bf6cc3b333ef12e8647c78515998fb9451d233eafdec37607d2d not found: ID does not exist"
Sep 10 12:20:45.841693 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.841669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:45.870921 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.870893    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dthg8\" (UniqueName: \"kubernetes.io/projected/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-kube-api-access-dthg8\") pod \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") "
Sep 10 12:20:45.871055 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.871022    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-maas-api-tls\") pod \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") "
Sep 10 12:20:45.871128 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.871081    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-metrics-certs\") pod \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") "
Sep 10 12:20:45.871128 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.871113    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-openshift-service-ca\") pod \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\" (UID: \"ceb2d558-7949-4aa3-a6d3-4b8cd68242bc\") "
Sep 10 12:20:45.871566 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.871533    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" (UID: "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:20:45.872978 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.872948    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-kube-api-access-dthg8" (OuterVolumeSpecName: "kube-api-access-dthg8") pod "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" (UID: "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc"). InnerVolumeSpecName "kube-api-access-dthg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:20:45.873086 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.873046    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" (UID: "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:20:45.873086 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.873065    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" (UID: "ceb2d558-7949-4aa3-a6d3-4b8cd68242bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:20:45.971839 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.971803    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dthg8\" (UniqueName: \"kubernetes.io/projected/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-kube-api-access-dthg8\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:20:45.971839 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.971833    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:20:45.971839 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.971846    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:20:45.972091 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:45.971858    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:20:46.145886 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:46.145809    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"]
Sep 10 12:20:46.151738 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:46.151711    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-2fc5d5-6db97c754d-rh2kl"]
Sep 10 12:20:46.158698 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:46.158673    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:47.892749 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:47.892714    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" path="/var/lib/kubelet/pods/ceb2d558-7949-4aa3-a6d3-4b8cd68242bc/volumes"
Sep 10 12:20:52.125136 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:52.125101    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:52.226324 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:52.226291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:52.407222 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:52.407122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:53.451403 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:53.451366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:20:56.263992 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:20:56.263961    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:11.597079 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.597043    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"]
Sep 10 12:21:11.597606 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.597252    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" podUID="4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" containerName="maas-api" containerID="cri-o://d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38" gracePeriod=30
Sep 10 12:21:11.629590 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.629559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:11.652327 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.652299    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"]
Sep 10 12:21:11.652522 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.652501    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4" podUID="10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" containerName="payload-processing" containerID="cri-o://7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75" gracePeriod=30
Sep 10 12:21:11.731945 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.731901    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:11.831702 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.831670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:11.894190 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.894139    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:11.897564 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.897528    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:21:11.953242 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.953213    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:21:11.973743 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.973705    2571 generic.go:356] "Generic (PLEG): container finished" podID="4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" containerID="d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38" exitCode=0
Sep 10 12:21:11.973936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.973767    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"
Sep 10 12:21:11.973936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.973867    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" event={"ID":"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac","Type":"ContainerDied","Data":"d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38"}
Sep 10 12:21:11.973936 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.973915    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg" event={"ID":"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac","Type":"ContainerDied","Data":"14a5263dd6f1549bf4e7b9f3db09eb998f7f6b8d404149e60c26ff908344758c"}
Sep 10 12:21:11.974094 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.973944    2571 scope.go:117] "RemoveContainer" containerID="d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38"
Sep 10 12:21:11.976525 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.976493    2571 generic.go:356] "Generic (PLEG): container finished" podID="10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" containerID="7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75" exitCode=0
Sep 10 12:21:11.976666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.976529    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4" event={"ID":"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a","Type":"ContainerDied","Data":"7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75"}
Sep 10 12:21:11.976666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.976552    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"
Sep 10 12:21:11.976666 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.976583    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4" event={"ID":"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a","Type":"ContainerDied","Data":"7d9bf7a95a516a29b584a03f8a714d5ae078e1f3d534bc7c92a4c8999f0768ae"}
Sep 10 12:21:11.983673 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.983654    2571 scope.go:117] "RemoveContainer" containerID="d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38"
Sep 10 12:21:11.983972 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:21:11.983944    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38\": container with ID starting with d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38 not found: ID does not exist" containerID="d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38"
Sep 10 12:21:11.984067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.983982    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38"} err="failed to get container status \"d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38\": rpc error: code = NotFound desc = could not find container \"d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38\": container with ID starting with d9ecfa2ba3fdff9ed9eee9cb162587acb061d4be87bd2c1a25955ea7798b9f38 not found: ID does not exist"
Sep 10 12:21:11.984067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.984004    2571 scope.go:117] "RemoveContainer" containerID="7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75"
Sep 10 12:21:11.992321 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.992301    2571 scope.go:117] "RemoveContainer" containerID="7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75"
Sep 10 12:21:11.992598 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:21:11.992579    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75\": container with ID starting with 7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75 not found: ID does not exist" containerID="7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75"
Sep 10 12:21:11.992662 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.992607    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75"} err="failed to get container status \"7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75\": rpc error: code = NotFound desc = could not find container \"7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75\": container with ID starting with 7f9ba7d35c33d74d297964ff05b6a467107e281c76c3dc3f2c731d8204451f75 not found: ID does not exist"
Sep 10 12:21:11.993861 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.993841    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98sqs\" (UniqueName: \"kubernetes.io/projected/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-kube-api-access-98sqs\") pod \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") "
Sep 10 12:21:11.993947 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.993902    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlwgf\" (UniqueName: \"kubernetes.io/projected/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-kube-api-access-mlwgf\") pod \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") "
Sep 10 12:21:11.993947 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.993937    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-openshift-service-ca\") pod \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") "
Sep 10 12:21:11.994058 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.993961    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-plugins-config-volume\") pod \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\" (UID: \"10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a\") "
Sep 10 12:21:11.994058 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.994024    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-metrics-certs\") pod \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") "
Sep 10 12:21:11.994058 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.994049    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls\") pod \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\" (UID: \"4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac\") "
Sep 10 12:21:11.994357 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.994329    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" (UID: "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:21:11.994426 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.994361    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" (UID: "10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:21:11.996043 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.996016    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-kube-api-access-mlwgf" (OuterVolumeSpecName: "kube-api-access-mlwgf") pod "10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" (UID: "10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a"). InnerVolumeSpecName "kube-api-access-mlwgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:21:11.996186 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.996148    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-kube-api-access-98sqs" (OuterVolumeSpecName: "kube-api-access-98sqs") pod "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" (UID: "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac"). InnerVolumeSpecName "kube-api-access-98sqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:21:11.996501 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.996478    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" (UID: "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:21:11.996591 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:11.996544    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" (UID: "4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:21:12.094747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.094713    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98sqs\" (UniqueName: \"kubernetes.io/projected/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-kube-api-access-98sqs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:21:12.094747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.094741    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlwgf\" (UniqueName: \"kubernetes.io/projected/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-kube-api-access-mlwgf\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:21:12.094747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.094750    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:21:12.094977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.094758    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:21:12.094977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.094768    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:21:12.094977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.094776    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:21:12.228592 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.228549    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:12.298593 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.298558    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"]
Sep 10 12:21:12.302905 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.302878    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-19f6c3-7ffbc77894-6xnzg"]
Sep 10 12:21:12.314872 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.314849    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"]
Sep 10 12:21:12.319365 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:12.319343    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-19f6c3-57bcc5d9cc-bmsc4"]
Sep 10 12:21:13.891988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:13.891945    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" path="/var/lib/kubelet/pods/10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a/volumes"
Sep 10 12:21:13.892643 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:13.892622    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" path="/var/lib/kubelet/pods/4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac/volumes"
Sep 10 12:21:17.298408 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:17.298368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:30.160744 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:30.160708    2571 scope.go:117] "RemoveContainer" containerID="ce70579b369f4df2c786acdd19d2d27d2e7abb184ad28f30520b325bcddff1e8"
Sep 10 12:21:40.072744 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:40.072710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:54.373523 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:54.373485    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:21:54.487217 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:21:54.487182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:00.341925 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:00.341888    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"]
Sep 10 12:22:00.342501 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:00.342231    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy" containerID="cri-o://0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375" gracePeriod=30
Sep 10 12:22:00.757127 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:00.757090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:00.872446 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:00.872413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:01.013313 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:01.013226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:05.590665 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.590638    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:22:05.672269 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672194    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-socket\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672269 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672226    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-data\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672269 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672248    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-envoy\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672284    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6006247-8ab1-4956-9450-0fac48291ce2-istio-podinfo\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672301    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6006247-8ab1-4956-9450-0fac48291ce2-istiod-ca-cert\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672330    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s55vx\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-kube-api-access-s55vx\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672372    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-istio-token\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672413    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-certs\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672446    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-credential-socket\") pod \"b6006247-8ab1-4956-9450-0fac48291ce2\" (UID: \"b6006247-8ab1-4956-9450-0fac48291ce2\") "
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672490    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:05.672543 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672507    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:05.672968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672702    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:05.672968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672753    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.672968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672773    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-data\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.672968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672859    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:05.672968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.672965    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6006247-8ab1-4956-9450-0fac48291ce2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:22:05.674550 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.674515    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:05.674663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.674596    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:05.674663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.674608    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-kube-api-access-s55vx" (OuterVolumeSpecName: "kube-api-access-s55vx") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "kube-api-access-s55vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:05.674663 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.674601    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6006247-8ab1-4956-9450-0fac48291ce2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6006247-8ab1-4956-9450-0fac48291ce2" (UID: "b6006247-8ab1-4956-9450-0fac48291ce2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 12:22:05.773331 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773298    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6006247-8ab1-4956-9450-0fac48291ce2-istio-podinfo\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.773331 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773324    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6006247-8ab1-4956-9450-0fac48291ce2-istiod-ca-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.773331 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773333    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s55vx\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-kube-api-access-s55vx\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.773551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773344    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6006247-8ab1-4956-9450-0fac48291ce2-istio-token\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.773551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773353    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-workload-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.773551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773360    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-credential-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:05.773551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:05.773367    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6006247-8ab1-4956-9450-0fac48291ce2-istio-envoy\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:06.281060 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.281022    2571 generic.go:356] "Generic (PLEG): container finished" podID="b6006247-8ab1-4956-9450-0fac48291ce2" containerID="0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375" exitCode=0
Sep 10 12:22:06.281060 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.281068    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" event={"ID":"b6006247-8ab1-4956-9450-0fac48291ce2","Type":"ContainerDied","Data":"0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375"}
Sep 10 12:22:06.281323 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.281107    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c" event={"ID":"b6006247-8ab1-4956-9450-0fac48291ce2","Type":"ContainerDied","Data":"e0109795a966c00481879ef87af4ef4592d4db3dee22b69f75f7baa2d0454253"}
Sep 10 12:22:06.281323 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.281104    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"
Sep 10 12:22:06.281323 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.281122    2571 scope.go:117] "RemoveContainer" containerID="0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375"
Sep 10 12:22:06.289988 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.289973    2571 scope.go:117] "RemoveContainer" containerID="461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"
Sep 10 12:22:06.298014 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.297996    2571 scope.go:117] "RemoveContainer" containerID="0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375"
Sep 10 12:22:06.298321 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:06.298300    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375\": container with ID starting with 0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375 not found: ID does not exist" containerID="0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375"
Sep 10 12:22:06.298379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.298329    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375"} err="failed to get container status \"0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375\": rpc error: code = NotFound desc = could not find container \"0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375\": container with ID starting with 0c662bd71dbb522e359fad36ea33a5aab44f6ca4fd74d5bee2e54d2c8aa35375 not found: ID does not exist"
Sep 10 12:22:06.298379 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.298346    2571 scope.go:117] "RemoveContainer" containerID="461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"
Sep 10 12:22:06.298598 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:06.298581    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a\": container with ID starting with 461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a not found: ID does not exist" containerID="461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"
Sep 10 12:22:06.298643 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.298602    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a"} err="failed to get container status \"461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a\": rpc error: code = NotFound desc = could not find container \"461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a\": container with ID starting with 461c49b8548c9dd75ee84ca8de36ca042413483c5fa07b2270e6ee4a1dbcd41a not found: ID does not exist"
Sep 10 12:22:06.304377 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.304354    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"]
Sep 10 12:22:06.306330 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:06.306312    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-19f6c3-openshift-default-6b556ff9db-vpg8c"]
Sep 10 12:22:07.892684 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:07.892651    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" path="/var/lib/kubelet/pods/b6006247-8ab1-4956-9450-0fac48291ce2/volumes"
Sep 10 12:22:11.286625 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:11.286600    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:11.430847 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:11.430820    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:11.567335 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:11.567218    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:11.691478 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:11.691448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:11.774917 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:11.774885    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:12.092249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:12.092213    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:19.075105 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:19.075073    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:19.182287 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:19.182252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:19.487696 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:19.487661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:20.158637 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:20.158605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:30.599837 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.599762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:30.653453 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.653421    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"]
Sep 10 12:22:30.653674 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.653634    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm" podUID="15223f0a-e868-4a11-93c8-aced51adeed2" containerName="payload-pre-processing" containerID="cri-o://a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd" gracePeriod=30
Sep 10 12:22:30.772427 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.772357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:30.865480 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.865350    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:30.914809 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.914784    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:22:30.959122 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.959094    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:30.987214 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.987174    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wqhc\" (UniqueName: \"kubernetes.io/projected/15223f0a-e868-4a11-93c8-aced51adeed2-kube-api-access-2wqhc\") pod \"15223f0a-e868-4a11-93c8-aced51adeed2\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") "
Sep 10 12:22:30.987438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.987299    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15223f0a-e868-4a11-93c8-aced51adeed2-plugins-config-volume\") pod \"15223f0a-e868-4a11-93c8-aced51adeed2\" (UID: \"15223f0a-e868-4a11-93c8-aced51adeed2\") "
Sep 10 12:22:30.987639 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.987615    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15223f0a-e868-4a11-93c8-aced51adeed2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15223f0a-e868-4a11-93c8-aced51adeed2" (UID: "15223f0a-e868-4a11-93c8-aced51adeed2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:22:30.989284 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:30.989260    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15223f0a-e868-4a11-93c8-aced51adeed2-kube-api-access-2wqhc" (OuterVolumeSpecName: "kube-api-access-2wqhc") pod "15223f0a-e868-4a11-93c8-aced51adeed2" (UID: "15223f0a-e868-4a11-93c8-aced51adeed2"). InnerVolumeSpecName "kube-api-access-2wqhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:31.088184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.088123    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wqhc\" (UniqueName: \"kubernetes.io/projected/15223f0a-e868-4a11-93c8-aced51adeed2-kube-api-access-2wqhc\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:31.088184 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.088180    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15223f0a-e868-4a11-93c8-aced51adeed2-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:31.305585 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.305552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:31.407217 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.407189    2571 generic.go:356] "Generic (PLEG): container finished" podID="15223f0a-e868-4a11-93c8-aced51adeed2" containerID="a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd" exitCode=0
Sep 10 12:22:31.407385 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.407231    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"
Sep 10 12:22:31.407385 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.407282    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm" event={"ID":"15223f0a-e868-4a11-93c8-aced51adeed2","Type":"ContainerDied","Data":"a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd"}
Sep 10 12:22:31.407385 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.407352    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm" event={"ID":"15223f0a-e868-4a11-93c8-aced51adeed2","Type":"ContainerDied","Data":"66706a31afc064fc1e417e371ba3620fe747f5386b05f902f9db32adcc9f8dab"}
Sep 10 12:22:31.407385 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.407374    2571 scope.go:117] "RemoveContainer" containerID="a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd"
Sep 10 12:22:31.416383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.416366    2571 scope.go:117] "RemoveContainer" containerID="a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd"
Sep 10 12:22:31.416645 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:31.416623    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd\": container with ID starting with a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd not found: ID does not exist" containerID="a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd"
Sep 10 12:22:31.416717 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.416651    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd"} err="failed to get container status \"a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd\": rpc error: code = NotFound desc = could not find container \"a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd\": container with ID starting with a561cfe8514a19d83219cb60d5c4798d3a0de594a97f60697943c09e0a6261cd not found: ID does not exist"
Sep 10 12:22:31.447768 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.447737    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"]
Sep 10 12:22:31.457955 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.457933    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c07be0-77578f5546-gr5cm"]
Sep 10 12:22:31.892323 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:31.892290    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15223f0a-e868-4a11-93c8-aced51adeed2" path="/var/lib/kubelet/pods/15223f0a-e868-4a11-93c8-aced51adeed2/volumes"
Sep 10 12:22:38.173498 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:38.173463    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:38.386616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:38.386581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:38.448362 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:38.448331    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"]
Sep 10 12:22:38.448554 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:38.448533    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy" containerID="cri-o://b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099" gracePeriod=30
Sep 10 12:22:38.480742 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:38.480716    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:39.026222 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:39.026186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:43.688215 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.688191    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:22:43.795726 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795639    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-credential-socket\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.795726 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795711    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-token\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.795949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795775    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-data\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.795949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795808    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-socket\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.795949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795849    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-podinfo\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.795949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795885    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istiod-ca-cert\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.795949 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795911    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-envoy\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.796249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-certs\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.796249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795978    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:43.796249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.795998    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bv25l\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-kube-api-access-bv25l\") pod \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\" (UID: \"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a\") "
Sep 10 12:22:43.796249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.796210    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:43.796463 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.796365    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:43.796463 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.796400    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-credential-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.796463 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.796423    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-data\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.796785 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.796759    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:43.796870 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.796834    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:22:43.798177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.798139    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:22:43.798330 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.798306    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 12:22:43.798423 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.798400    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:43.798507 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.798489    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-kube-api-access-bv25l" (OuterVolumeSpecName: "kube-api-access-bv25l") pod "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" (UID: "aa4e3672-c7f4-49d4-aa51-c2ed03041a9a"). InnerVolumeSpecName "kube-api-access-bv25l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:43.897249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897217    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-socket\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.897249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897240    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-podinfo\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.897249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897248    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istiod-ca-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.897249 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897256    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-envoy\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.897497 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897264    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-workload-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.897497 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897272    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bv25l\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-kube-api-access-bv25l\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:43.897497 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:43.897281    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a-istio-token\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:44.475849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.475808    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerID="b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099" exitCode=0
Sep 10 12:22:44.476034 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.475866    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"
Sep 10 12:22:44.476034 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.475901    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" event={"ID":"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a","Type":"ContainerDied","Data":"b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099"}
Sep 10 12:22:44.476034 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.475949    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl" event={"ID":"aa4e3672-c7f4-49d4-aa51-c2ed03041a9a","Type":"ContainerDied","Data":"e78e3b6ed97011daded1402f544b3c05502e946a4a2b3946ab5de321d80dbdb1"}
Sep 10 12:22:44.476034 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.475979    2571 scope.go:117] "RemoveContainer" containerID="b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099"
Sep 10 12:22:44.484701 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.484683    2571 scope.go:117] "RemoveContainer" containerID="9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"
Sep 10 12:22:44.492580 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.492560    2571 scope.go:117] "RemoveContainer" containerID="b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099"
Sep 10 12:22:44.492878 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:44.492857    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099\": container with ID starting with b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099 not found: ID does not exist" containerID="b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099"
Sep 10 12:22:44.492964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.492885    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099"} err="failed to get container status \"b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099\": rpc error: code = NotFound desc = could not find container \"b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099\": container with ID starting with b095ae90b3dc18360a1837ee9418b9ecc0d6ccebc76fa56b61b8a2c0427b4099 not found: ID does not exist"
Sep 10 12:22:44.492964 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.492905    2571 scope.go:117] "RemoveContainer" containerID="9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"
Sep 10 12:22:44.493186 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:44.493168    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402\": container with ID starting with 9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402 not found: ID does not exist" containerID="9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"
Sep 10 12:22:44.493242 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.493192    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402"} err="failed to get container status \"9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402\": rpc error: code = NotFound desc = could not find container \"9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402\": container with ID starting with 9b366e2a14b370ec5451d68b3c5af0bc5fb49bd3b259b3cc0eb17a8e4fc8d402 not found: ID does not exist"
Sep 10 12:22:44.501517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.501494    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"]
Sep 10 12:22:44.503482 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:44.503463    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-c07be0-openshift-default-84986b8846-rnsvl"]
Sep 10 12:22:45.893234 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:45.893199    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" path="/var/lib/kubelet/pods/aa4e3672-c7f4-49d4-aa51-c2ed03041a9a/volumes"
Sep 10 12:22:49.571862 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.571831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:49.644787 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.644754    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"]
Sep 10 12:22:49.645018 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.644980    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82" podUID="c247afe7-848d-4786-a345-03cff25809d4" containerName="payload-processing" containerID="cri-o://6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e" gracePeriod=30
Sep 10 12:22:49.647002 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.646976    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"]
Sep 10 12:22:49.647213 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.647189    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89" podUID="9c151b8e-2ff0-4668-aa73-c7f992ca55fc" containerName="payload-pre-processing" containerID="cri-o://abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99" gracePeriod=30
Sep 10 12:22:49.694602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.694570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:49.806162 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.806109    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:49.919999 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.919975    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:22:49.922817 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.922801    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:22:49.932131 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.932111    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:49.946405 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946383    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9wxk\" (UniqueName: \"kubernetes.io/projected/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-kube-api-access-l9wxk\") pod \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") "
Sep 10 12:22:49.946517 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946441    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c247afe7-848d-4786-a345-03cff25809d4-plugins-config-volume\") pod \"c247afe7-848d-4786-a345-03cff25809d4\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") "
Sep 10 12:22:49.946623 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946606    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-plugins-config-volume\") pod \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\" (UID: \"9c151b8e-2ff0-4668-aa73-c7f992ca55fc\") "
Sep 10 12:22:49.946678 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946634    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4ffd\" (UniqueName: \"kubernetes.io/projected/c247afe7-848d-4786-a345-03cff25809d4-kube-api-access-b4ffd\") pod \"c247afe7-848d-4786-a345-03cff25809d4\" (UID: \"c247afe7-848d-4786-a345-03cff25809d4\") "
Sep 10 12:22:49.946782 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946756    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c247afe7-848d-4786-a345-03cff25809d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c247afe7-848d-4786-a345-03cff25809d4" (UID: "c247afe7-848d-4786-a345-03cff25809d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:22:49.946926 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946907    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c247afe7-848d-4786-a345-03cff25809d4-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:49.946991 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.946959    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9c151b8e-2ff0-4668-aa73-c7f992ca55fc" (UID: "9c151b8e-2ff0-4668-aa73-c7f992ca55fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:22:49.948594 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.948572    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-kube-api-access-l9wxk" (OuterVolumeSpecName: "kube-api-access-l9wxk") pod "9c151b8e-2ff0-4668-aa73-c7f992ca55fc" (UID: "9c151b8e-2ff0-4668-aa73-c7f992ca55fc"). InnerVolumeSpecName "kube-api-access-l9wxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:49.948659 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:49.948610    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c247afe7-848d-4786-a345-03cff25809d4-kube-api-access-b4ffd" (OuterVolumeSpecName: "kube-api-access-b4ffd") pod "c247afe7-848d-4786-a345-03cff25809d4" (UID: "c247afe7-848d-4786-a345-03cff25809d4"). InnerVolumeSpecName "kube-api-access-b4ffd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:22:50.046136 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.046106    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:50.048325 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.048307    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-plugins-config-volume\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:50.048398 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.048328    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4ffd\" (UniqueName: \"kubernetes.io/projected/c247afe7-848d-4786-a345-03cff25809d4-kube-api-access-b4ffd\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:50.048398 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.048339    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9wxk\" (UniqueName: \"kubernetes.io/projected/9c151b8e-2ff0-4668-aa73-c7f992ca55fc-kube-api-access-l9wxk\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:22:50.348370 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.348338    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:50.508596 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.508562    2571 generic.go:356] "Generic (PLEG): container finished" podID="9c151b8e-2ff0-4668-aa73-c7f992ca55fc" containerID="abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99" exitCode=0
Sep 10 12:22:50.508770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.508604    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"
Sep 10 12:22:50.508770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.508648    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89" event={"ID":"9c151b8e-2ff0-4668-aa73-c7f992ca55fc","Type":"ContainerDied","Data":"abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99"}
Sep 10 12:22:50.508770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.508703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89" event={"ID":"9c151b8e-2ff0-4668-aa73-c7f992ca55fc","Type":"ContainerDied","Data":"3bdb1913ee64789b7d7bc20a563712752d7dbf7156eb97eb52491ebc2aabf6f5"}
Sep 10 12:22:50.508770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.508731    2571 scope.go:117] "RemoveContainer" containerID="abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99"
Sep 10 12:22:50.510258 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.510235    2571 generic.go:356] "Generic (PLEG): container finished" podID="c247afe7-848d-4786-a345-03cff25809d4" containerID="6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e" exitCode=0
Sep 10 12:22:50.510352 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.510282    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"
Sep 10 12:22:50.510352 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.510323    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82" event={"ID":"c247afe7-848d-4786-a345-03cff25809d4","Type":"ContainerDied","Data":"6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e"}
Sep 10 12:22:50.510453 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.510368    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82" event={"ID":"c247afe7-848d-4786-a345-03cff25809d4","Type":"ContainerDied","Data":"108fa73af4bbadabbc63a0e58bafa417fbae360d9760efe93584669bfc646eed"}
Sep 10 12:22:50.519177 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.519136    2571 scope.go:117] "RemoveContainer" containerID="abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99"
Sep 10 12:22:50.519453 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:50.519430    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99\": container with ID starting with abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99 not found: ID does not exist" containerID="abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99"
Sep 10 12:22:50.519539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.519462    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99"} err="failed to get container status \"abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99\": rpc error: code = NotFound desc = could not find container \"abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99\": container with ID starting with abbd2459a2346bf135027e830f1bfb484d86d21c4fc1b9c95dd3321313096a99 not found: ID does not exist"
Sep 10 12:22:50.519539 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.519487    2571 scope.go:117] "RemoveContainer" containerID="6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e"
Sep 10 12:22:50.527361 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.527344    2571 scope.go:117] "RemoveContainer" containerID="6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e"
Sep 10 12:22:50.527596 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:22:50.527574    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e\": container with ID starting with 6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e not found: ID does not exist" containerID="6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e"
Sep 10 12:22:50.527657 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.527605    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e"} err="failed to get container status \"6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e\": rpc error: code = NotFound desc = could not find container \"6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e\": container with ID starting with 6583f8c03c19f8a160dba14821bd2ed1d6255e177fb3deaa742e308d3803ff9e not found: ID does not exist"
Sep 10 12:22:50.540563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.540543    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"]
Sep 10 12:22:50.548871 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.548848    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7ed5ac-9dd87ff8b-f5x89"]
Sep 10 12:22:50.568609 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.568586    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"]
Sep 10 12:22:50.572654 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:50.572635    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7ed5ac-68d895cc55-pzn82"]
Sep 10 12:22:51.892692 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:51.892656    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c151b8e-2ff0-4668-aa73-c7f992ca55fc" path="/var/lib/kubelet/pods/9c151b8e-2ff0-4668-aa73-c7f992ca55fc/volumes"
Sep 10 12:22:51.893078 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:51.892996    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c247afe7-848d-4786-a345-03cff25809d4" path="/var/lib/kubelet/pods/c247afe7-848d-4786-a345-03cff25809d4/volumes"
Sep 10 12:22:56.886934 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:56.886883    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:56.988382 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:56.988325    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:57.205587 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:57.205557    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:22:59.876218 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:59.876177    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-4kmsh"]
Sep 10 12:22:59.876639 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:22:59.876375    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh" podUID="951bbd11-849c-4cb7-8b44-559629964f98" containerName="manager" containerID="cri-o://d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd" gracePeriod=30
Sep 10 12:23:00.212083 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.212060    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:23:00.330551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.330521    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxcmf\" (UniqueName: \"kubernetes.io/projected/951bbd11-849c-4cb7-8b44-559629964f98-kube-api-access-qxcmf\") pod \"951bbd11-849c-4cb7-8b44-559629964f98\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") "
Sep 10 12:23:00.330736 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.330560    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs\") pod \"951bbd11-849c-4cb7-8b44-559629964f98\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") "
Sep 10 12:23:00.330736 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.330614    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-webhook-cert\") pod \"951bbd11-849c-4cb7-8b44-559629964f98\" (UID: \"951bbd11-849c-4cb7-8b44-559629964f98\") "
Sep 10 12:23:00.332508 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.332475    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951bbd11-849c-4cb7-8b44-559629964f98-kube-api-access-qxcmf" (OuterVolumeSpecName: "kube-api-access-qxcmf") pod "951bbd11-849c-4cb7-8b44-559629964f98" (UID: "951bbd11-849c-4cb7-8b44-559629964f98"). InnerVolumeSpecName "kube-api-access-qxcmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:23:00.332637 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.332619    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "951bbd11-849c-4cb7-8b44-559629964f98" (UID: "951bbd11-849c-4cb7-8b44-559629964f98"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:23:00.332693 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.332636    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "951bbd11-849c-4cb7-8b44-559629964f98" (UID: "951bbd11-849c-4cb7-8b44-559629964f98"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:23:00.431778 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.431757    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxcmf\" (UniqueName: \"kubernetes.io/projected/951bbd11-849c-4cb7-8b44-559629964f98-kube-api-access-qxcmf\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:23:00.431913 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.431779    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:23:00.431913 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.431788    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951bbd11-849c-4cb7-8b44-559629964f98-webhook-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:23:00.567433 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.567399    2571 generic.go:356] "Generic (PLEG): container finished" podID="951bbd11-849c-4cb7-8b44-559629964f98" containerID="d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd" exitCode=0
Sep 10 12:23:00.567616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.567447    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh"
Sep 10 12:23:00.567616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.567487    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh" event={"ID":"951bbd11-849c-4cb7-8b44-559629964f98","Type":"ContainerDied","Data":"d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd"}
Sep 10 12:23:00.567616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.567563    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-4kmsh" event={"ID":"951bbd11-849c-4cb7-8b44-559629964f98","Type":"ContainerDied","Data":"2aa437a25e1af69bba09f407d32314ce8cddd9b9eed36dc42f333e491625c91f"}
Sep 10 12:23:00.567616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.567587    2571 scope.go:117] "RemoveContainer" containerID="d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd"
Sep 10 12:23:00.577240 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.577221    2571 scope.go:117] "RemoveContainer" containerID="d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd"
Sep 10 12:23:00.577518 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:23:00.577502    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd\": container with ID starting with d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd not found: ID does not exist" containerID="d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd"
Sep 10 12:23:00.577584 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.577528    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd"} err="failed to get container status \"d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd\": rpc error: code = NotFound desc = could not find container \"d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd\": container with ID starting with d0462a3ad5b23f33442b8bdde61028b9f1cd8e8d7867ac4f7823a1cd709c8ddd not found: ID does not exist"
Sep 10 12:23:00.590440 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.590413    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-4kmsh"]
Sep 10 12:23:00.592849 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:00.592828    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-4kmsh"]
Sep 10 12:23:01.892611 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:01.892568    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="951bbd11-849c-4cb7-8b44-559629964f98" path="/var/lib/kubelet/pods/951bbd11-849c-4cb7-8b44-559629964f98/volumes"
Sep 10 12:23:37.261262 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:37.261225    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:23:40.103697 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:23:40.103666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:24:01.408725 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.408682    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-htz45"]
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409093    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15223f0a-e868-4a11-93c8-aced51adeed2" containerName="payload-pre-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409112    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="15223f0a-e868-4a11-93c8-aced51adeed2" containerName="payload-pre-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409127    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c247afe7-848d-4786-a345-03cff25809d4" containerName="payload-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409137    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c247afe7-848d-4786-a345-03cff25809d4" containerName="payload-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409147    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409173    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409182    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409190    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409197    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" containerName="maas-api"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409204    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" containerName="maas-api"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409221    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409229    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409238    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" containerName="maas-api"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409243    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" containerName="maas-api"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409249    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="951bbd11-849c-4cb7-8b44-559629964f98" containerName="manager"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409254    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="951bbd11-849c-4cb7-8b44-559629964f98" containerName="manager"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409261    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c151b8e-2ff0-4668-aa73-c7f992ca55fc" containerName="payload-pre-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409266    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c151b8e-2ff0-4668-aa73-c7f992ca55fc" containerName="payload-pre-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409276    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409281    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409289    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" containerName="payload-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409294    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" containerName="payload-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409352    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c247afe7-848d-4786-a345-03cff25809d4" containerName="payload-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409360    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409366    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6006247-8ab1-4956-9450-0fac48291ce2" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409373    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="10e19f8a-bccc-4a55-bb5c-c9569b8a5d7a" containerName="payload-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409379    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="15223f0a-e868-4a11-93c8-aced51adeed2" containerName="payload-pre-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409385    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="951bbd11-849c-4cb7-8b44-559629964f98" containerName="manager"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409391    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0a5107-b8e7-45a7-bbdf-3259f2e9f6ac" containerName="maas-api"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409397    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c151b8e-2ff0-4668-aa73-c7f992ca55fc" containerName="payload-pre-processing"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409403    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409407    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb2d558-7949-4aa3-a6d3-4b8cd68242bc" containerName="maas-api"
Sep 10 12:24:01.411317 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.409413    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa4e3672-c7f4-49d4-aa51-c2ed03041a9a" containerName="istio-proxy"
Sep 10 12:24:01.412606 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.412230    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.416233 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.416206    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 12:24:01.417200 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.417173    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 12:24:01.417523 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.417505    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-smn6x\""
Sep 10 12:24:01.417925 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.417901    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 12:24:01.427813 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.427784    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-htz45"]
Sep 10 12:24:01.512902 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.512861    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-metrics-certs\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.513101 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.512915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-webhook-cert\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.513101 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.512933    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwfcs\" (UniqueName: \"kubernetes.io/projected/a3afb788-4fb4-4269-aa77-7165743969b6-kube-api-access-hwfcs\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.613695 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.613651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-webhook-cert\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.613910 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.613701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwfcs\" (UniqueName: \"kubernetes.io/projected/a3afb788-4fb4-4269-aa77-7165743969b6-kube-api-access-hwfcs\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.613910 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.613797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-metrics-certs\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.616225 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.616196    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-webhook-cert\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.616354 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.616280    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-metrics-certs\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.623873 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.623852    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwfcs\" (UniqueName: \"kubernetes.io/projected/a3afb788-4fb4-4269-aa77-7165743969b6-kube-api-access-hwfcs\") pod \"maas-controller-6c88dd9b7f-htz45\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") " pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:01.725067 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:01.725020    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:02.003110 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:02.003025    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-htz45"]
Sep 10 12:24:02.006908 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:24:02.006883    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3afb788_4fb4_4269_aa77_7165743969b6.slice/crio-d39eb910930fee396137a5fb4b07fb772287816f651517296375978d9660060b WatchSource:0}: Error finding container d39eb910930fee396137a5fb4b07fb772287816f651517296375978d9660060b: Status 404 returned error can't find the container with id d39eb910930fee396137a5fb4b07fb772287816f651517296375978d9660060b
Sep 10 12:24:02.872425 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:02.872339    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-htz45" event={"ID":"a3afb788-4fb4-4269-aa77-7165743969b6","Type":"ContainerStarted","Data":"3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4"}
Sep 10 12:24:02.872425 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:02.872398    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-htz45" event={"ID":"a3afb788-4fb4-4269-aa77-7165743969b6","Type":"ContainerStarted","Data":"d39eb910930fee396137a5fb4b07fb772287816f651517296375978d9660060b"}
Sep 10 12:24:02.872820 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:02.872518    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:02.901207 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:02.901132    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c88dd9b7f-htz45" podStartSLOduration=1.433023215 podStartE2EDuration="1.901117209s" totalImagesPullingTime="468.093994ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:02.897705551 +0000 UTC m=+1953.500980463" watchObservedRunningTime="2026-09-10 12:24:02.901117209 +0000 UTC m=+1953.504392120"
Sep 10 12:24:05.827826 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:05.827780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:24:06.013491 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:06.013455    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:24:13.879700 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:13.879670    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:22.170510 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:22.170432    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:24:23.200306 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.200276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:24:23.291979 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.291938    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-htz45"]
Sep 10 12:24:23.292274 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.292245    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c88dd9b7f-htz45" podUID="a3afb788-4fb4-4269-aa77-7165743969b6" containerName="manager" containerID="cri-o://3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4" gracePeriod=30
Sep 10 12:24:23.538860 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.538828    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:23.591074 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.591041    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-metrics-certs\") pod \"a3afb788-4fb4-4269-aa77-7165743969b6\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") "
Sep 10 12:24:23.591284 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.591132    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwfcs\" (UniqueName: \"kubernetes.io/projected/a3afb788-4fb4-4269-aa77-7165743969b6-kube-api-access-hwfcs\") pod \"a3afb788-4fb4-4269-aa77-7165743969b6\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") "
Sep 10 12:24:23.591284 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.591193    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-webhook-cert\") pod \"a3afb788-4fb4-4269-aa77-7165743969b6\" (UID: \"a3afb788-4fb4-4269-aa77-7165743969b6\") "
Sep 10 12:24:23.593660 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.593615    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a3afb788-4fb4-4269-aa77-7165743969b6" (UID: "a3afb788-4fb4-4269-aa77-7165743969b6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:24:23.593836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.593803    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3afb788-4fb4-4269-aa77-7165743969b6-kube-api-access-hwfcs" (OuterVolumeSpecName: "kube-api-access-hwfcs") pod "a3afb788-4fb4-4269-aa77-7165743969b6" (UID: "a3afb788-4fb4-4269-aa77-7165743969b6"). InnerVolumeSpecName "kube-api-access-hwfcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:24:23.594296 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.594255    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a3afb788-4fb4-4269-aa77-7165743969b6" (UID: "a3afb788-4fb4-4269-aa77-7165743969b6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:24:23.692541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.692499    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwfcs\" (UniqueName: \"kubernetes.io/projected/a3afb788-4fb4-4269-aa77-7165743969b6-kube-api-access-hwfcs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:24:23.692541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.692536    2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-webhook-cert\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:24:23.692541 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.692549    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3afb788-4fb4-4269-aa77-7165743969b6-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:24:23.971608 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.971571    2571 generic.go:356] "Generic (PLEG): container finished" podID="a3afb788-4fb4-4269-aa77-7165743969b6" containerID="3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4" exitCode=0
Sep 10 12:24:23.971754 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.971611    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-htz45"
Sep 10 12:24:23.971754 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.971640    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-htz45" event={"ID":"a3afb788-4fb4-4269-aa77-7165743969b6","Type":"ContainerDied","Data":"3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4"}
Sep 10 12:24:23.971754 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.971685    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-htz45" event={"ID":"a3afb788-4fb4-4269-aa77-7165743969b6","Type":"ContainerDied","Data":"d39eb910930fee396137a5fb4b07fb772287816f651517296375978d9660060b"}
Sep 10 12:24:23.971754 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.971709    2571 scope.go:117] "RemoveContainer" containerID="3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4"
Sep 10 12:24:23.980144 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.980126    2571 scope.go:117] "RemoveContainer" containerID="3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4"
Sep 10 12:24:23.980440 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:24:23.980415    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4\": container with ID starting with 3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4 not found: ID does not exist" containerID="3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4"
Sep 10 12:24:23.980535 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:23.980455    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4"} err="failed to get container status \"3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4\": rpc error: code = NotFound desc = could not find container \"3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4\": container with ID starting with 3c3b42ccf48cb920d7d53ef84251a3e8fa7908913e40d376534a62f585d99af4 not found: ID does not exist"
Sep 10 12:24:24.009627 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.009589    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-htz45"]
Sep 10 12:24:24.015166 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.015122    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-htz45"]
Sep 10 12:24:24.748118 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.748083    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:24:24.821624 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.821589    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-547pf"]
Sep 10 12:24:24.821913 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.821900    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3afb788-4fb4-4269-aa77-7165743969b6" containerName="manager"
Sep 10 12:24:24.821913 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.821914    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3afb788-4fb4-4269-aa77-7165743969b6" containerName="manager"
Sep 10 12:24:24.822001 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.821988    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3afb788-4fb4-4269-aa77-7165743969b6" containerName="manager"
Sep 10 12:24:24.825897 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.825876    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:24.829866 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.829798    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-smn6x\""
Sep 10 12:24:24.829866 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.829816    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 12:24:24.830062 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.829866    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 12:24:24.830062 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.829933    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 12:24:24.842466 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.842442    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-547pf"]
Sep 10 12:24:24.903602 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.903561    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-webhook-cert\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:24.903828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.903612    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-metrics-certs\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:24.903828 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:24.903680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s7gj\" (UniqueName: \"kubernetes.io/projected/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-kube-api-access-5s7gj\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.004605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.004502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-webhook-cert\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.004605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.004551    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-metrics-certs\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.004605 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.004571    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s7gj\" (UniqueName: \"kubernetes.io/projected/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-kube-api-access-5s7gj\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.007068 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.007033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-metrics-certs\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.007220 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.007195    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-webhook-cert\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.018996 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.018966    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s7gj\" (UniqueName: \"kubernetes.io/projected/7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d-kube-api-access-5s7gj\") pod \"maas-controller-6c88dd9b7f-547pf\" (UID: \"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d\") " pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.136369 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.136327    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:25.418202 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.418177    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c88dd9b7f-547pf"]
Sep 10 12:24:25.420049 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:24:25.420021    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7002bf4e_efd1_4b70_8e8b_cb9a6ce9250d.slice/crio-c3d21cf299f4ae95ba38483b47d7cd41bd71fee08ab7810e207e4a508b1b2887 WatchSource:0}: Error finding container c3d21cf299f4ae95ba38483b47d7cd41bd71fee08ab7810e207e4a508b1b2887: Status 404 returned error can't find the container with id c3d21cf299f4ae95ba38483b47d7cd41bd71fee08ab7810e207e4a508b1b2887
Sep 10 12:24:25.891824 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.891788    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3afb788-4fb4-4269-aa77-7165743969b6" path="/var/lib/kubelet/pods/a3afb788-4fb4-4269-aa77-7165743969b6/volumes"
Sep 10 12:24:25.982797 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:25.982770    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-547pf" event={"ID":"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d","Type":"ContainerStarted","Data":"c3d21cf299f4ae95ba38483b47d7cd41bd71fee08ab7810e207e4a508b1b2887"}
Sep 10 12:24:26.989619 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:26.989586    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c88dd9b7f-547pf" event={"ID":"7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d","Type":"ContainerStarted","Data":"399aa229c2e1e59e5e60e39b7389bc92689911d624a3dcea1933ef8d639de37b"}
Sep 10 12:24:26.990069 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:26.989686    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:24:27.013135 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:27.013076    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c88dd9b7f-547pf" podStartSLOduration=2.4710393379999998 podStartE2EDuration="3.013058248s" totalImagesPullingTime="542.01891ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:24:27.012114856 +0000 UTC m=+1977.615389792" watchObservedRunningTime="2026-09-10 12:24:27.013058248 +0000 UTC m=+1977.616333159"
Sep 10 12:24:37.997478 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:24:37.997447    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c88dd9b7f-547pf"
Sep 10 12:25:23.324710 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:25:23.324672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:26:41.970183 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:26:41.970136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:26:42.073690 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:26:42.073658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:27:32.330753 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:32.330674    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:27:39.375638 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.375605    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"]
Sep 10 12:27:39.378802 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.378780    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.381457 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.381434    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-4b778336-kserve-self-signed-certs\""
Sep 10 12:27:39.392753 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.392728    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"]
Sep 10 12:27:39.423581 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.423546    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:27:39.540341 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.540303    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zl8b\" (UniqueName: \"kubernetes.io/projected/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-kube-api-access-6zl8b\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.540513 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.540354    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-dshm\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.540513 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.540437    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tls-certs\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.540513 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.540479    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-model-cache\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.540513 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.540503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-home\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.540713 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.540580    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tmp-dir\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.641747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.641645    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-dshm\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.641747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.641706    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tls-certs\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.641747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.641727    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-model-cache\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.641747 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.641748    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-home\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.642077 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.641777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tmp-dir\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.642077 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.641822    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zl8b\" (UniqueName: \"kubernetes.io/projected/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-kube-api-access-6zl8b\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.642216 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.642141    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-model-cache\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.642279 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.642225    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tmp-dir\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.642336 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.642271    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-home\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.643968 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.643947    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-dshm\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.644340 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.644321    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tls-certs\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.650979 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.650953    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zl8b\" (UniqueName: \"kubernetes.io/projected/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-kube-api-access-6zl8b\") pod \"e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") " pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.688678 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.688653    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:39.962725 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:39.962700    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"]
Sep 10 12:27:39.964953 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:27:39.964928    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0e8e2dc_9950_4ceb_99fa_4005506aa6b0.slice/crio-a14da4ca96e4b7036614ca102c995f83eb4dcc7430252fb936b487ce5bf5170d WatchSource:0}: Error finding container a14da4ca96e4b7036614ca102c995f83eb4dcc7430252fb936b487ce5bf5170d: Status 404 returned error can't find the container with id a14da4ca96e4b7036614ca102c995f83eb4dcc7430252fb936b487ce5bf5170d
Sep 10 12:27:40.930548 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:40.930513    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4" event={"ID":"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0","Type":"ContainerStarted","Data":"03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6"}
Sep 10 12:27:40.930959 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:40.930559    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4" event={"ID":"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0","Type":"ContainerStarted","Data":"a14da4ca96e4b7036614ca102c995f83eb4dcc7430252fb936b487ce5bf5170d"}
Sep 10 12:27:49.689246 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:49.689200    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:49.689246 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:49.689253    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:49.707866 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:49.707833    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:27:49.737168 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:49.737105    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4" podStartSLOduration=10.737091115 podStartE2EDuration="10.737091115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:27:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:27:40.960348612 +0000 UTC m=+2171.563623551" watchObservedRunningTime="2026-09-10 12:27:49.737091115 +0000 UTC m=+2180.340366025"
Sep 10 12:27:49.984635 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:27:49.984560    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:28:00.888226 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:00.888189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:28:01.976755 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:01.976722    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"]
Sep 10 12:28:02.130093 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:02.130060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:28:16.996224 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:16.996179    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4" podUID="a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" containerName="main" containerID="cri-o://03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6" gracePeriod=45
Sep 10 12:28:17.238448 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.238420    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:28:17.304438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304360    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tmp-dir\") pod \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") "
Sep 10 12:28:17.304438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304395    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-model-cache\") pod \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") "
Sep 10 12:28:17.304438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304415    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-dshm\") pod \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") "
Sep 10 12:28:17.304438 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304433    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-home\") pod \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") "
Sep 10 12:28:17.304774 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304493    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zl8b\" (UniqueName: \"kubernetes.io/projected/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-kube-api-access-6zl8b\") pod \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") "
Sep 10 12:28:17.304774 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304521    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tls-certs\") pod \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\" (UID: \"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0\") "
Sep 10 12:28:17.304774 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304689    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" (UID: "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:28:17.304774 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304724    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" (UID: "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:28:17.304922 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.304770    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-home" (OuterVolumeSpecName: "home") pod "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" (UID: "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:28:17.306563 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.306528    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" (UID: "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:28:17.306676 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.306576    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-dshm" (OuterVolumeSpecName: "dshm") pod "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" (UID: "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 12:28:17.306676 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.306611    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-kube-api-access-6zl8b" (OuterVolumeSpecName: "kube-api-access-6zl8b") pod "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" (UID: "a0e8e2dc-9950-4ceb-99fa-4005506aa6b0"). InnerVolumeSpecName "kube-api-access-6zl8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:28:17.405240 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.405206    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tls-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:17.405383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.405250    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-tmp-dir\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:17.405383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.405259    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-model-cache\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:17.405383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.405267    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-dshm\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:17.405383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.405274    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-home\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:17.405383 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:17.405283    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zl8b\" (UniqueName: \"kubernetes.io/projected/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0-kube-api-access-6zl8b\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:18.110613 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.110576    2571 generic.go:356] "Generic (PLEG): container finished" podID="a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" containerID="03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6" exitCode=0
Sep 10 12:28:18.111004 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.110618    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"
Sep 10 12:28:18.111004 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.110617    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4" event={"ID":"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0","Type":"ContainerDied","Data":"03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6"}
Sep 10 12:28:18.111004 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.110725    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4" event={"ID":"a0e8e2dc-9950-4ceb-99fa-4005506aa6b0","Type":"ContainerDied","Data":"a14da4ca96e4b7036614ca102c995f83eb4dcc7430252fb936b487ce5bf5170d"}
Sep 10 12:28:18.111004 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.110742    2571 scope.go:117] "RemoveContainer" containerID="03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6"
Sep 10 12:28:18.121293 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.121273    2571 scope.go:117] "RemoveContainer" containerID="03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6"
Sep 10 12:28:18.121731 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:28:18.121713    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6\": container with ID starting with 03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6 not found: ID does not exist" containerID="03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6"
Sep 10 12:28:18.121794 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.121740    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6"} err="failed to get container status \"03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6\": rpc error: code = NotFound desc = could not find container \"03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6\": container with ID starting with 03d82b3b8d0b3dda93a58bd769f1818685ab6d89e939f55461855df4b883ffa6 not found: ID does not exist"
Sep 10 12:28:18.162742 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.162682    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"]
Sep 10 12:28:18.165977 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:18.165953    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-4b778336-kserve-554cb85b95-c8js4"]
Sep 10 12:28:19.892556 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:19.892520    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" path="/var/lib/kubelet/pods/a0e8e2dc-9950-4ceb-99fa-4005506aa6b0/volumes"
Sep 10 12:28:20.383850 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.383809    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"]
Sep 10 12:28:20.384238 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.384218    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" containerName="main"
Sep 10 12:28:20.384346 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.384240    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" containerName="main"
Sep 10 12:28:20.384346 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.384309    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0e8e2dc-9950-4ceb-99fa-4005506aa6b0" containerName="main"
Sep 10 12:28:20.388836 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.388814    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.403775 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.403749    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"]
Sep 10 12:28:20.427294 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.427268    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/576f351f-e649-4fe2-9ec5-4562b58c73b3-metrics-certs\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.427393 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.427333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576f351f-e649-4fe2-9ec5-4562b58c73b3-maas-api-tls\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.427393 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.427360    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk9vc\" (UniqueName: \"kubernetes.io/projected/576f351f-e649-4fe2-9ec5-4562b58c73b3-kube-api-access-dk9vc\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.427495 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.427397    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576f351f-e649-4fe2-9ec5-4562b58c73b3-openshift-service-ca\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.528534 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.528508    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576f351f-e649-4fe2-9ec5-4562b58c73b3-maas-api-tls\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.528534 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.528539    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk9vc\" (UniqueName: \"kubernetes.io/projected/576f351f-e649-4fe2-9ec5-4562b58c73b3-kube-api-access-dk9vc\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.528775 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.528562    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576f351f-e649-4fe2-9ec5-4562b58c73b3-openshift-service-ca\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.528775 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.528589    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/576f351f-e649-4fe2-9ec5-4562b58c73b3-metrics-certs\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.529293 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.529274    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576f351f-e649-4fe2-9ec5-4562b58c73b3-openshift-service-ca\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.531041 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.531012    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/576f351f-e649-4fe2-9ec5-4562b58c73b3-metrics-certs\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.531128 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.531081    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576f351f-e649-4fe2-9ec5-4562b58c73b3-maas-api-tls\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.539479 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.539453    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk9vc\" (UniqueName: \"kubernetes.io/projected/576f351f-e649-4fe2-9ec5-4562b58c73b3-kube-api-access-dk9vc\") pod \"maas-api-645cf68f76-8s2r2\" (UID: \"576f351f-e649-4fe2-9ec5-4562b58c73b3\") " pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.698576 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.698545    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:20.978707 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.978678    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"]
Sep 10 12:28:20.980929 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:28:20.980901    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576f351f_e649_4fe2_9ec5_4562b58c73b3.slice/crio-33d57af11ed4e6b5c8fe4fccbea798f3c1267791bccab20260f180d426ff73cd WatchSource:0}: Error finding container 33d57af11ed4e6b5c8fe4fccbea798f3c1267791bccab20260f180d426ff73cd: Status 404 returned error can't find the container with id 33d57af11ed4e6b5c8fe4fccbea798f3c1267791bccab20260f180d426ff73cd
Sep 10 12:28:20.982388 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:20.982365    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 12:28:21.127551 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:21.127516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2" event={"ID":"576f351f-e649-4fe2-9ec5-4562b58c73b3","Type":"ContainerStarted","Data":"33d57af11ed4e6b5c8fe4fccbea798f3c1267791bccab20260f180d426ff73cd"}
Sep 10 12:28:22.133854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:22.133814    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2" event={"ID":"576f351f-e649-4fe2-9ec5-4562b58c73b3","Type":"ContainerStarted","Data":"9b0e3d0e8274d99d7a4c5521ac944942993fddc893c5bd750bee97fec0cab67e"}
Sep 10 12:28:22.134263 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:22.133902    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:22.155366 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:22.155322    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2" podStartSLOduration=1.606702761 podStartE2EDuration="2.155308419s" totalImagesPullingTime="548.605658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 12:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:28:22.154398173 +0000 UTC m=+2212.757673085" watchObservedRunningTime="2026-09-10 12:28:22.155308419 +0000 UTC m=+2212.758583330"
Sep 10 12:28:28.144274 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:28.144246    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-645cf68f76-8s2r2"
Sep 10 12:28:28.200107 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:28.200075    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"]
Sep 10 12:28:28.200328 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:28.200302    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp" podUID="46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" containerName="maas-api" containerID="cri-o://ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a" gracePeriod=30
Sep 10 12:28:29.439753 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.439726    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:28:29.502242 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.502206    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lplqc\" (UniqueName: \"kubernetes.io/projected/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-kube-api-access-lplqc\") pod \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") "
Sep 10 12:28:29.502452 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.502286    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-metrics-certs\") pod \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") "
Sep 10 12:28:29.502452 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.502337    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-openshift-service-ca\") pod \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") "
Sep 10 12:28:29.502452 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.502374    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-maas-api-tls\") pod \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\" (UID: \"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8\") "
Sep 10 12:28:29.502769 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.502733    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" (UID: "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 12:28:29.504395 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.504373    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" (UID: "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:28:29.504928 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.504872    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" (UID: "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 12:28:29.504928 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.504886    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-kube-api-access-lplqc" (OuterVolumeSpecName: "kube-api-access-lplqc") pod "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" (UID: "46aecbb6-c487-47e3-88a6-c6f5adb9f2f8"). InnerVolumeSpecName "kube-api-access-lplqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 12:28:29.603087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.603043    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-openshift-service-ca\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:29.603087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.603079    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-maas-api-tls\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:29.603087 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.603092    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lplqc\" (UniqueName: \"kubernetes.io/projected/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-kube-api-access-lplqc\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:29.603361 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:29.603104    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8-metrics-certs\") on node \"ip-10-0-129-41.ec2.internal\" DevicePath \"\""
Sep 10 12:28:30.176839 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.176740    2571 generic.go:356] "Generic (PLEG): container finished" podID="46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" containerID="ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a" exitCode=0
Sep 10 12:28:30.176839 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.176778    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"
Sep 10 12:28:30.176839 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.176822    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp" event={"ID":"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8","Type":"ContainerDied","Data":"ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a"}
Sep 10 12:28:30.177094 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.176877    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp" event={"ID":"46aecbb6-c487-47e3-88a6-c6f5adb9f2f8","Type":"ContainerDied","Data":"167e452b6757eff985e79eb463aace463e232dd3d440296f10a60797cdc0a975"}
Sep 10 12:28:30.177094 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.176902    2571 scope.go:117] "RemoveContainer" containerID="ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a"
Sep 10 12:28:30.185556 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.185539    2571 scope.go:117] "RemoveContainer" containerID="ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a"
Sep 10 12:28:30.185849 ip-10-0-129-41 kubenswrapper[2571]: E0910 12:28:30.185831    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a\": container with ID starting with ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a not found: ID does not exist" containerID="ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a"
Sep 10 12:28:30.185909 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.185858    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a"} err="failed to get container status \"ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a\": rpc error: code = NotFound desc = could not find container \"ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a\": container with ID starting with ef1c31a8758a88d150bb9e771acbfcfbecbee6ee9617a09546440af8e88cda0a not found: ID does not exist"
Sep 10 12:28:30.197692 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.197669    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"]
Sep 10 12:28:30.202888 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:30.202865    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bb5f968cb-n6qnp"]
Sep 10 12:28:31.891853 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:31.891818    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" path="/var/lib/kubelet/pods/46aecbb6-c487-47e3-88a6-c6f5adb9f2f8/volumes"
Sep 10 12:28:56.063966 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:56.063881    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c88dd9b7f-547pf_7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d/manager/0.log"
Sep 10 12:28:56.687109 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:56.687083    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86dfbd7c-xjg9v_4df95b75-8775-4c56-a5f7-7db80b0c956f/copy-manifests/0.log"
Sep 10 12:28:56.694097 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:56.694076    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86dfbd7c-xjg9v_4df95b75-8775-4c56-a5f7-7db80b0c956f/manager/0.log"
Sep 10 12:28:57.039662 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:57.039581    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-645cf68f76-8s2r2_576f351f-e649-4fe2-9ec5-4562b58c73b3/maas-api/0.log"
Sep 10 12:28:59.316765 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:59.316734    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd_05ac0d9e-4686-4505-8faf-0ffe1cd91361/istio-proxy/0.log"
Sep 10 12:28:59.437101 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:28:59.437068    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qnbd7_622567f6-4603-45ce-92c2-a2fc38a99e88/discovery/0.log"
Sep 10 12:29:00.035440 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:00.035406    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scdzb_b7713abc-e4fa-4b21-8453-a6fd6ad7a824/payload-processing/0.log"
Sep 10 12:29:01.087450 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:01.087407    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6f292_459764d3-5850-4079-9aa9-a9bec7b196ab/main/0.log"
Sep 10 12:29:06.761429 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:06.761396    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:29:08.321032 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:08.321001    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:29:09.869217 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:09.869189    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nkxp2_92c58a92-66b4-4673-887e-b84c82a92d80/global-pull-secret-syncer/0.log"
Sep 10 12:29:09.926770 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:09.926742    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log"
Sep 10 12:29:10.020382 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:10.020348    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-41.ec2.internal_2bac53f559f1c749c3a586b3ce98b441/haproxy/0.log"
Sep 10 12:29:16.817820 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:16.817790    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-nkzjm_04d8a797-3778-4a8c-becf-401dd09db8d3/cluster-monitoring-operator/0.log"
Sep 10 12:29:16.945108 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:16.945072    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b7c6fc9cb-4b6tl_fff471a9-5e58-4d5c-92de-da6f253c51b0/metrics-server/0.log"
Sep 10 12:29:17.258958 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:17.258925    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kn6d7_07bdaa2f-0f8e-416d-b5ab-c6d71d161c99/node-exporter/0.log"
Sep 10 12:29:17.291428 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:17.291402    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kn6d7_07bdaa2f-0f8e-416d-b5ab-c6d71d161c99/kube-rbac-proxy/0.log"
Sep 10 12:29:17.316043 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:17.316020    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kn6d7_07bdaa2f-0f8e-416d-b5ab-c6d71d161c99/init-textfile/0.log"
Sep 10 12:29:19.016725 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.016689    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lxrm_1e193539-5ca1-4471-b03a-edba883854ad/konnectivity-agent/0.log" limit=1024
Sep 10 12:29:19.278190 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.278084    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"]
Sep 10 12:29:19.278524 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.278506    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" containerName="maas-api"
Sep 10 12:29:19.278616 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.278528    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" containerName="maas-api"
Sep 10 12:29:19.278675 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.278623    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="46aecbb6-c487-47e3-88a6-c6f5adb9f2f8" containerName="maas-api"
Sep 10 12:29:19.281460 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.281439    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.286187 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.286166    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xpjf\"/\"kube-root-ca.crt\""
Sep 10 12:29:19.290111 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.290092    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xpjf\"/\"openshift-service-ca.crt\""
Sep 10 12:29:19.290864 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.290848    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8xpjf\"/\"default-dockercfg-k9vmw\""
Sep 10 12:29:19.297637 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.297612    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"]
Sep 10 12:29:19.312909 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.312887    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-podres\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.313016 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.312919    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-sys\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.313016 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.312990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgqw7\" (UniqueName: \"kubernetes.io/projected/2c13f7ea-99ec-437e-80eb-9aa710e18252-kube-api-access-lgqw7\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.313110 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.313022    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-proc\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.313110 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.313044    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-lib-modules\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414064 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414031    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-podres\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414265 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414084    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-sys\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414265 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqw7\" (UniqueName: \"kubernetes.io/projected/2c13f7ea-99ec-437e-80eb-9aa710e18252-kube-api-access-lgqw7\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414265 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414242    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-sys\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414263    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-proc\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-proc\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414295    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-lib-modules\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414242    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-podres\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.414399 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.414360    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c13f7ea-99ec-437e-80eb-9aa710e18252-lib-modules\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.425201 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.425174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgqw7\" (UniqueName: \"kubernetes.io/projected/2c13f7ea-99ec-437e-80eb-9aa710e18252-kube-api-access-lgqw7\") pod \"perf-node-gather-daemonset-jrnhz\" (UID: \"2c13f7ea-99ec-437e-80eb-9aa710e18252\") " pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.591536 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.591442    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:19.869581 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:19.869555    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"]
Sep 10 12:29:19.872292 ip-10-0-129-41 kubenswrapper[2571]: W0910 12:29:19.872259    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2c13f7ea_99ec_437e_80eb_9aa710e18252.slice/crio-824858d3f0c5901b94d87b916712de09c7a5c7c5cfa1acea99467db3ed12d5d7 WatchSource:0}: Error finding container 824858d3f0c5901b94d87b916712de09c7a5c7c5cfa1acea99467db3ed12d5d7: Status 404 returned error can't find the container with id 824858d3f0c5901b94d87b916712de09c7a5c7c5cfa1acea99467db3ed12d5d7
Sep 10 12:29:20.419652 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:20.419606    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz" event={"ID":"2c13f7ea-99ec-437e-80eb-9aa710e18252","Type":"ContainerStarted","Data":"284353a8f24b15165dc9f5b1231dd3db41f82129e9bc0dff5270b106c19d174c"}
Sep 10 12:29:20.420066 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:20.419667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz" event={"ID":"2c13f7ea-99ec-437e-80eb-9aa710e18252","Type":"ContainerStarted","Data":"824858d3f0c5901b94d87b916712de09c7a5c7c5cfa1acea99467db3ed12d5d7"}
Sep 10 12:29:20.420066 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:20.419697    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:20.451861 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:20.446815    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz" podStartSLOduration=1.446780048 podStartE2EDuration="1.446780048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 12:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 12:29:20.444602378 +0000 UTC m=+2271.047877288" watchObservedRunningTime="2026-09-10 12:29:20.446780048 +0000 UTC m=+2271.050054960"
Sep 10 12:29:22.806854 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:22.806827    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f4dmp_738ddeb2-5f6a-4545-bc93-b7bcca8e3836/dns/0.log"
Sep 10 12:29:22.833538 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:22.833512    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f4dmp_738ddeb2-5f6a-4545-bc93-b7bcca8e3836/kube-rbac-proxy/0.log"
Sep 10 12:29:22.890688 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:22.890656    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v4ghr_89662635-b5c4-47b7-8e37-b30f4b58fda4/dns-node-resolver/0.log"
Sep 10 12:29:23.548488 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:23.548456    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dxl2z_0b8668d6-db4b-43c5-8567-51fcb8429334/node-ca/0.log"
Sep 10 12:29:24.593889 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:24.593862    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd_05ac0d9e-4686-4505-8faf-0ffe1cd91361/istio-proxy/0.log"
Sep 10 12:29:24.828166 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:24.828128    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qnbd7_622567f6-4603-45ce-92c2-a2fc38a99e88/discovery/0.log"
Sep 10 12:29:25.051488 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:25.051459    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scdzb_b7713abc-e4fa-4b21-8453-a6fd6ad7a824/payload-processing/0.log"
Sep 10 12:29:25.963695 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:25.963666    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4z47p_fe9a7703-a40d-4b5d-91dd-5ff448d80136/serve-healthcheck-canary/0.log"
Sep 10 12:29:26.433863 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:26.433837    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8xpjf/perf-node-gather-daemonset-jrnhz"
Sep 10 12:29:26.629521 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:26.629491    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pcjzx_671603a8-2ec0-4abd-ac85-cd068f477e7e/kube-rbac-proxy/0.log"
Sep 10 12:29:26.654580 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:26.654556    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pcjzx_671603a8-2ec0-4abd-ac85-cd068f477e7e/exporter/0.log"
Sep 10 12:29:26.680490 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:26.680466    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pcjzx_671603a8-2ec0-4abd-ac85-cd068f477e7e/extractor/0.log"
Sep 10 12:29:29.592741 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:29.592713    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c88dd9b7f-547pf_7002bf4e-efd1-4b70-8e8b-cb9a6ce9250d/manager/0.log"
Sep 10 12:29:29.920331 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:29.920246    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86dfbd7c-xjg9v_4df95b75-8775-4c56-a5f7-7db80b0c956f/manager/0.log"
Sep 10 12:29:29.943149 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:29.943119    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86dfbd7c-xjg9v_4df95b75-8775-4c56-a5f7-7db80b0c956f/copy-manifests/0.log"
Sep 10 12:29:30.405252 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:30.405218    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd_05ac0d9e-4686-4505-8faf-0ffe1cd91361/istio-proxy/0.log"
Sep 10 12:29:30.627349 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:30.627327    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qnbd7_622567f6-4603-45ce-92c2-a2fc38a99e88/discovery/0.log"
Sep 10 12:29:30.748537 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:30.748491    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scdzb_b7713abc-e4fa-4b21-8453-a6fd6ad7a824/payload-processing/0.log"
Sep 10 12:29:31.374934 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:31.374902    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5f7f7d474c-6xt7d_e8fa3e7f-a6a7-4c3d-845a-c9cc0861cf77/manager/0.log"
Sep 10 12:29:32.246295 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:32.246269    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c2n4nd_05ac0d9e-4686-4505-8faf-0ffe1cd91361/istio-proxy/0.log"
Sep 10 12:29:32.463352 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:32.463323    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-qnbd7_622567f6-4603-45ce-92c2-a2fc38a99e88/discovery/0.log"
Sep 10 12:29:32.590618 ip-10-0-129-41 kubenswrapper[2571]: I0910 12:29:32.590548    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-scdzb_b7713abc-e4fa-4b21-8453-a6fd6ad7a824/payload-processing/0.log"