Jul 29 00:52:27.045387 ip-10-0-139-6 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 00:52:27.045397 ip-10-0-139-6 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 00:52:27.045404 ip-10-0-139-6 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 00:52:27.045636 ip-10-0-139-6 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 00:52:37.275683 ip-10-0-139-6 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 00:52:37.275699 ip-10-0-139-6 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c87af89aec7f42688fc4aaf62b04da67 --
Jul 29 00:54:29.183397 ip-10-0-139-6 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 00:54:29.575948 ip-10-0-139-6 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:54:29.575948 ip-10-0-139-6 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 00:54:29.575948 ip-10-0-139-6 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:54:29.575948 ip-10-0-139-6 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 00:54:29.575948 ip-10-0-139-6 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 00:54:29.576634 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.575997    2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 00:54:29.582394 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582379    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:29.582394 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582395    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582399    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582403    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582406    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582409    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582412    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582415    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582419    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582424    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582432    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582435    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582438    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582440    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582443    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582446    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582449    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582451    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582454    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582456    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582459    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:29.582461 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582462    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582465    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582468    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582471    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582474    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582476    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582479    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582482    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582484    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582486    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582489    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582491    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582494    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582499    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582502    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582505    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582508    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582511    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582514    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:29.582931 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582517    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582519    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582522    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582524    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582526    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582529    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582531    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582534    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582536    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582539    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582541    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582544    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582546    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582549    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582551    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582555    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582559    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582562    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582565    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582568    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:29.583426 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582571    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582576    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582579    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582581    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582584    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582586    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582589    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582591    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582594    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582596    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582599    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582603    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582606    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582608    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582610    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582613    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582615    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582618    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582620    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582623    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:29.583908 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582626    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582628    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582631    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582633    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582636    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582638    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582640    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582644    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582647    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582650    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582652    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582655    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582658    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582660    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582663    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582665    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582668    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582671    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:29.584402 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.582673    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:29.585031 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585017    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:29.585031 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585029    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585033    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585036    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585039    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585042    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585044    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585047    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585050    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585053    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585056    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585058    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585061    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585063    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585066    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585068    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585071    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585074    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585076    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585079    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585081    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:29.585090 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585084    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585087    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585089    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585092    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585095    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585098    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585102    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585106    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585109    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585112    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585115    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585117    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585120    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585123    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585125    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585128    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585142    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585146    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585148    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:29.585598 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585151    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585154    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585157    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585161    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585165    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585168    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585171    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585174    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585176    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585179    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585182    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585184    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585187    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585189    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585192    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585195    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585198    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585201    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585203    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:29.586047 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585206    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585210    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585213    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585216    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585219    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585221    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585224    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585226    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585229    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585232    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585234    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585236    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585239    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585242    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585244    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585247    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585250    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585253    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585256    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585258    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:29.586517 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585261    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585263    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585266    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585268    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585271    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585273    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585276    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585278    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585281    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585284    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585287    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585289    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585292    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585295    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585297    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585300    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585303    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585305    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585308    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585310    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:29.586994 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585698    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585703    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585706    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585709    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585717    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585721    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585724    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585727    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585729    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585732    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585734    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585737    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585739    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585742    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585744    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585747    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585749    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585752    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585755    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:29.587597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585757    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585761    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585763    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585766    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585769    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585771    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585774    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585776    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585779    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585781    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585784    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585787    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585789    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585792    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585800    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585803    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585806    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585808    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585812    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585815    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:29.588061 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585817    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585820    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585823    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585826    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585830    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585833    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585837    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585839    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585842    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585845    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585848    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585850    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585853    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585855    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585858    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585861    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585863    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585866    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585868    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:29.588587 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585870    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585873    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585875    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585878    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585881    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585883    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585885    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585889    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585892    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585894    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585896    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585899    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585902    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585904    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585907    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585909    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585911    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585914    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585917    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585919    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:29.589066 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585923    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585926    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585929    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585932    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585934    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585937    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585940    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585942    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585945    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585947    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585950    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585953    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585956    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585958    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585961    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585963    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585966    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585968    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585971    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585975    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:29.589566 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.585977    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586041    2555 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586053    2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586060    2555 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586064    2555 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586069    2555 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586073    2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586077    2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586081    2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586084    2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586088    2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586091    2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586095    2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586098    2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586101    2555 flags.go:64] FLAG: --cgroup-root=""
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586104    2555 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586106    2555 flags.go:64] FLAG: --client-ca-file=""
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586110    2555 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586113    2555 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586117    2555 flags.go:64] FLAG: --cluster-domain=""
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586119    2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586123    2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586126    2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586129    2555 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 00:54:29.590085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586147    2555 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586150    2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586153    2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586156    2555 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586159    2555 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586162    2555 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586165    2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586169    2555 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586172    2555 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586176    2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586179    2555 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586182    2555 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586185    2555 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586188    2555 flags.go:64] FLAG: --enable-server="true"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586195    2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586201    2555 flags.go:64] FLAG: --event-burst="100"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586205    2555 flags.go:64] FLAG: --event-qps="50"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586208    2555 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586210    2555 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586213    2555 flags.go:64] FLAG: --eviction-hard=""
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586217    2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586220    2555 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586223    2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586229    2555 flags.go:64] FLAG: --eviction-soft=""
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586232    2555 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 00:54:29.590680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586235    2555 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586238    2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586241    2555 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586244    2555 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586247    2555 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586250    2555 flags.go:64] FLAG: --feature-gates=""
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586254    2555 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586257    2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586260    2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586263    2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586266    2555 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586269    2555 flags.go:64] FLAG: --help="false"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586272    2555 flags.go:64] FLAG: --hostname-override="ip-10-0-139-6.ec2.internal"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586275    2555 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586278    2555 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586281    2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586284    2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586287    2555 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586290    2555 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586293    2555 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586296    2555 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586298    2555 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586302    2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586305    2555 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 00:54:29.591320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586308    2555 flags.go:64] FLAG: --kube-reserved=""
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586311    2555 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586313    2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586316    2555 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586319    2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586322    2555 flags.go:64] FLAG: --lock-file=""
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586325    2555 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586328    2555 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586331    2555 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586336    2555 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586339    2555 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586342    2555 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586345    2555 flags.go:64] FLAG: --logging-format="text"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586348    2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586351    2555 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586354    2555 flags.go:64] FLAG: --manifest-url=""
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586357    2555 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586361    2555 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586364    2555 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586368    2555 flags.go:64] FLAG: --max-pods="110"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586371    2555 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586374    2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586377    2555 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586380    2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586383    2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 00:54:29.591905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586386    2555 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586389    2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586399    2555 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586402    2555 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586405    2555 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586408    2555 flags.go:64] FLAG: --pod-cidr=""
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586411    2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586417    2555 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586420    2555 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586423    2555 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586426    2555 flags.go:64] FLAG: --port="10250"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586429    2555 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586432    2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01735acd05275e31f"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586435    2555 flags.go:64] FLAG: --qos-reserved=""
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586439    2555 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586442    2555 flags.go:64] FLAG: --register-node="true"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586445    2555 flags.go:64] FLAG: --register-with-taints=""
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586449    2555 flags.go:64] FLAG: --registry-burst="10"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586452    2555 flags.go:64] FLAG: --registry-qps="5"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586454    2555 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586457    2555 flags.go:64] FLAG: --reserved-memory=""
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586461    2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586464    2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586467    2555 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586470    2555 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 00:54:29.592521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586473    2555 flags.go:64] FLAG: --runonce="false"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586476    2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586479    2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586482    2555 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586484    2555 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586487    2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586490    2555 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586493    2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586496    2555 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586499    2555 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586502    2555 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586505    2555 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586507    2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586510    2555 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586514    2555 flags.go:64] FLAG: --system-cgroups=""
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586516    2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586522    2555 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586525    2555 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586528    2555 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586534    2555 flags.go:64] FLAG: --tls-min-version=""
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586537    2555 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586541    2555 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586544    2555 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586547    2555 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586550    2555 flags.go:64] FLAG: --v="2"
Jul 29 00:54:29.593189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586554    2555 flags.go:64] FLAG: --version="false"
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586558    2555 flags.go:64] FLAG: --vmodule=""
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586563    2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.586566    2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586683    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586687    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586690    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586693    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586696    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586699    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586702    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586705    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586707    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586710    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586713    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586715    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586719    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586723    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586726    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586729    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:29.593823 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586732    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586734    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586737    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586740    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586743    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586746    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586749    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586752    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586756    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586758    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586761    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586764    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586766    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586769    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586773    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586775    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586778    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586781    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586784    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:29.594347 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586787    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586790    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586793    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586795    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586798    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586800    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586803    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586805    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586808    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586811    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586813    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586816    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586819    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586821    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586824    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586826    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586832    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586835    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586838    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:29.594802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586840    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586843    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586847    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586850    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586852    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586855    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586857    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586860    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586864    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586866    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586869    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586872    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586874    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586877    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586879    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586882    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586884    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586887    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586890    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:29.595288 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586892    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586896    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586899    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586902    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586905    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586908    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586910    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586913    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586916    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586919    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586921    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586929    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586932    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586935    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586938    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586942    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586945    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586947    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586950    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586952    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:29.595744 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586955    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586960    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586962    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586965    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586967    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.586970    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.587779    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.594456    2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.594470    2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.595313    2555 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.595321    2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595385    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595390    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595394    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:29.596241 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595397    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595402    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595405    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595409    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595412    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595414    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595417    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595420    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595422    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595426    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595428    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595431    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595434    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595436    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595439    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595443    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595446    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595448    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595451    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:29.596595 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595454    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595456    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595459    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595462    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595465    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595467    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595470    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595475    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595478    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595481    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595484    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595487    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595489    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595492    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595495    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595498    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595500    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595503    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595506    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:29.597058 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595508    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595511    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595513    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595516    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595519    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595522    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595524    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595527    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595530    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595532    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595535    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595538    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595540    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595543    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595546    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595548    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595551    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595554    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595558    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595560    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:29.597532 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595564    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595567    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595570    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595573    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595575    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595578    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595581    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595583    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595586    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595588    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595590    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595593    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595596    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595598    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595601    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595603    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595606    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595615    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595618    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595621    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:29.598017 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595624    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595627    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595629    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595632    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595635    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595638    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595640    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595643    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595646    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595648    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595651    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595653    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595657    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595660    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595662    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595665    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595668    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:29.598529 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595671    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.595676    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595787    2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595791    2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595794    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595798    2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595801    2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595803    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595806    2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595809    2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595812    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595814    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595817    2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595820    2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595823    2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 00:54:29.598934 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595826    2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595828    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595831    2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595834    2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595837    2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595839    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595842    2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595844    2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595847    2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595850    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595852    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595855    2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595858    2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595861    2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595863    2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595866    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595868    2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595871    2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595874    2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595876    2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 00:54:29.599329 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595879    2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595883    2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595886    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595889    2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595892    2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595895    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595897    2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595900    2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595903    2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595905    2555 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595908    2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595911    2555 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595913    2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595916    2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595918    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595921    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595923    2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595926    2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595928    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 00:54:29.599802 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595931    2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595933    2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595935    2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595938    2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595941    2555 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595943    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595946    2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595949    2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595951    2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595953    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595956    2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595958    2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595961    2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595963    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595966    2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595968    2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595971    2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595973    2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595976    2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595979    2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 00:54:29.600275 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595982    2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595985    2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595987    2555 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595990    2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595992    2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595995    2555 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.595998    2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596000    2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596002    2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596005    2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596008    2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596010    2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596012    2555 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596015    2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596017    2555 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596020    2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596022    2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596025    2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596027    2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596030    2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 00:54:29.600756 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596034    2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596037    2555 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596040    2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596043    2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596045    2555 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596048    2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:29.596050    2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.596055    2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.596169    2555 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.598233    2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.599539    2555 server.go:1019] "Starting client certificate rotation"
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.599638    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 00:54:29.601270 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.599675    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 00:54:29.604606 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.604577    2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 00:54:29.606089 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.606072    2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 00:54:29.617521 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.617502    2555 log.go:25] "Validated CRI v1 runtime API"
Jul 29 00:54:29.619602 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.619588    2555 log.go:25] "Validated CRI v1 image API"
Jul 29 00:54:29.621198 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.621182    2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 00:54:29.625360 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.625339    2555 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c5eb099f-e6c3-4d7c-be43-1a39e0050775:/dev/nvme0n1p3 d279daf0-7851-47ca-adf7-9bbe74cf879f:/dev/nvme0n1p4]
Jul 29 00:54:29.625447 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.625360    2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 00:54:29.631343 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.631230    2555 manager.go:216] Machine: {Timestamp:2026-07-29 00:54:29.629930678 +0000 UTC m=+0.332722732 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102030 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2115de9dac23c607c382e8bd839bac SystemUUID:ec2115de-9dac-23c6-07c3-82e8bd839bac BootID:c87af89a-ec7f-4268-8fc4-aaf62b04da67 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b2:f9:8c:20:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b2:f9:8c:20:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:19:0d:3e:29:24 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 00:54:29.631343 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.631334    2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 00:54:29.631505 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.631489    2555 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 00:54:29.631676 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.631660    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 00:54:29.635561 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.635539    2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 00:54:29.635694 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.635564    2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 00:54:29.636329 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.636318    2555 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 00:54:29.636363 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.636331    2555 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 00:54:29.636363 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.636338    2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 00:54:29.636938 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.636922    2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 00:54:29.637770 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.637759    2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 00:54:29.640425 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640413    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 00:54:29.640533 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640524    2555 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 00:54:29.640566 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640553    2555 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 00:54:29.640566 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640564    2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 00:54:29.640629 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640578    2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 00:54:29.640629 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640587    2555 kubelet.go:398] "Adding apiserver pod source"
Jul 29 00:54:29.640629 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640597    2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 00:54:29.640960 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640947    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 00:54:29.641043 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.640968    2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 00:54:29.644395 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.644374    2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 00:54:29.646523 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.646506    2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 00:54:29.646611 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.646544    2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 00:54:29.648238 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648223    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648243    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648252    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648260    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648272    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648280    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648292    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648300    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 00:54:29.648309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648310    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 00:54:29.648558 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648319    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 00:54:29.648558 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648336    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 00:54:29.648558 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.648344    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 00:54:29.649159 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.649148    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 00:54:29.649216 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.649163    2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 00:54:29.651202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.651163    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k96bf"
Jul 29 00:54:29.651559 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.651540    2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 00:54:29.652235 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.652210    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 00:54:29.652235 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.652211    2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 00:54:29.652359 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.652352    2555 server.go:1268] "Started kubelet"
Jul 29 00:54:29.652510 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.652457    2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 00:54:29.652567 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.652528    2555 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 00:54:29.652605 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.652486    2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 00:54:29.653220 ip-10-0-139-6 systemd[1]: Started Kubernetes Kubelet.
Jul 29 00:54:29.654483 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.654467    2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 00:54:29.654782 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.654771    2555 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 00:54:29.656237 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.656218    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k96bf"
Jul 29 00:54:29.657663 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.657646    2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 00:54:29.663668 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.663654    2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 00:54:29.663735 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.663722    2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 00:54:29.663790 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.663739    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:29.664384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664367    2555 factory.go:55] Registering systemd factory
Jul 29 00:54:29.664461 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664386    2555 factory.go:223] Registration of the systemd container factory successfully
Jul 29 00:54:29.664517 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664490    2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 00:54:29.664517 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664510    2555 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 00:54:29.664602 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664525    2555 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 00:54:29.664602 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664590    2555 factory.go:153] Registering CRI-O factory
Jul 29 00:54:29.664602 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664603    2555 factory.go:223] Registration of the crio container factory successfully
Jul 29 00:54:29.664741 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664652    2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 00:54:29.664741 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664676    2555 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 00:54:29.664741 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664687    2555 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 00:54:29.664741 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664678    2555 factory.go:103] Registering Raw factory
Jul 29 00:54:29.664741 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.664721    2555 manager.go:1196] Started watching for new ooms in manager
Jul 29 00:54:29.665606 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.665365    2555 manager.go:349] Starting recovery of all containers
Jul 29 00:54:29.666346 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.665953    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:29.670916 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.670893    2555 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-6.ec2.internal\" not found" node="ip-10-0-139-6.ec2.internal"
Jul 29 00:54:29.678055 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.678040    2555 manager.go:354] Recovery completed
Jul 29 00:54:29.682002 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.681990    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:29.684194 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684176    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:29.684272 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684203    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:29.684272 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684214    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:29.684743 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684726    2555 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 00:54:29.684743 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684738    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:29.684743 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684741    2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 00:54:29.684864 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.684761    2555 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 00:54:29.685377 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.685362    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:29.685462 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.685389    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:29.685462 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.685402    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:29.687144 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.687123    2555 policy_none.go:49] "None policy: Start"
Jul 29 00:54:29.687186 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.687150    2555 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 00:54:29.687186 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.687160    2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 00:54:29.688845 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.688834    2555 policy_none.go:47] "Start"
Jul 29 00:54:29.737781 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.737662    2555 manager.go:337] "Starting Device Plugin manager"
Jul 29 00:54:29.737874 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.737817    2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 00:54:29.737874 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.737829    2555 server.go:87] "Starting device plugin registration server"
Jul 29 00:54:29.738032 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.738022    2555 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 00:54:29.738071 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.738033    2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 00:54:29.738277 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.738246    2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 00:54:29.738373 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.738334    2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 00:54:29.738373 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.738344    2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 00:54:29.739296 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.739204    2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 00:54:29.739296 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.739255    2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:29.809596 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.809570    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 00:54:29.811446 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.810766    2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 00:54:29.811446 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.810808    2555 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 00:54:29.811446 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.810833    2555 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 00:54:29.811446 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.810870    2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 00:54:29.812670 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.812652    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:29.838662 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.838581    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:29.839389 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.839366    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:29.839462 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.839407    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:29.839462 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.839422    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:29.839462 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.839449    2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-6.ec2.internal"
Jul 29 00:54:29.848256 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:29.848240    2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-6.ec2.internal"
Jul 29 00:54:29.848303 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:29.848259    2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-6.ec2.internal\": node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.234592 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.234576    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.283935 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.283898    2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"]
Jul 29 00:54:31.284072 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.283988    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:31.285607 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.285591    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:31.285681 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.285625    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:31.285681 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.285636    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:31.287720 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.287707    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:31.287843 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.287829    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.287926 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.287878    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:31.288430 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.288403    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:31.288534 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.288435    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:31.288534 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.288446    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:31.288534 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.288403    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:31.288534 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.288507    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:31.288534 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.288528    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:31.290540 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.290523    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.290629 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.290551    2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 00:54:31.291876 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.291862    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 00:54:31.291973 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.291891    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 00:54:31.291973 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.291905    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeHasSufficientPID"
Jul 29 00:54:31.313866 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.313849    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-6.ec2.internal\" not found" node="ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.318546 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.318531    2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-6.ec2.internal\" not found" node="ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.335001 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.334979    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.338086 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.338062    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c6b9a321efae5b6e2a235b013867e21-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal\" (UID: \"1c6b9a321efae5b6e2a235b013867e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.338150 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.338104    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c6b9a321efae5b6e2a235b013867e21-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal\" (UID: \"1c6b9a321efae5b6e2a235b013867e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.338187 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.338166    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df7a3dfe02b1862a6c7a987e8d3facb8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-6.ec2.internal\" (UID: \"df7a3dfe02b1862a6c7a987e8d3facb8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.435592 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.435555    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.438830 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.438805    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c6b9a321efae5b6e2a235b013867e21-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal\" (UID: \"1c6b9a321efae5b6e2a235b013867e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.438943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.438848    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c6b9a321efae5b6e2a235b013867e21-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal\" (UID: \"1c6b9a321efae5b6e2a235b013867e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.438943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.438911    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df7a3dfe02b1862a6c7a987e8d3facb8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-6.ec2.internal\" (UID: \"df7a3dfe02b1862a6c7a987e8d3facb8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.438943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.438910    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1c6b9a321efae5b6e2a235b013867e21-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal\" (UID: \"1c6b9a321efae5b6e2a235b013867e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.439058 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.438947    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/df7a3dfe02b1862a6c7a987e8d3facb8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-6.ec2.internal\" (UID: \"df7a3dfe02b1862a6c7a987e8d3facb8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.439058 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.438980    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c6b9a321efae5b6e2a235b013867e21-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal\" (UID: \"1c6b9a321efae5b6e2a235b013867e21\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.536311 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.536220    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.616682 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.616656    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.621925 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.621791    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:31.636593 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.636572    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.737082 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.737045    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.837536 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.837470    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.937947 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:31.937907    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:31.971219 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.971191    2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 00:54:31.971688 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.971350    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:54:31.971688 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:31.971361    2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 00:54:32.034193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.034154    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 00:49:31 +0000 UTC" deadline="2028-01-05 22:32:49.872090998 +0000 UTC"
Jul 29 00:54:32.034193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.034189    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12621h38m17.837905479s"
Jul 29 00:54:32.038005 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:32.037987    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:32.138717 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:32.138659    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:32.200591 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:32.200563    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf7a3dfe02b1862a6c7a987e8d3facb8.slice/crio-23fdd926f53da89e53d0f80350d84ebebc5f61ce0422b5f7a9b1521096e7c321 WatchSource:0}: Error finding container 23fdd926f53da89e53d0f80350d84ebebc5f61ce0422b5f7a9b1521096e7c321: Status 404 returned error can't find the container with id 23fdd926f53da89e53d0f80350d84ebebc5f61ce0422b5f7a9b1521096e7c321
Jul 29 00:54:32.201001 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:32.200985    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6b9a321efae5b6e2a235b013867e21.slice/crio-3b147771034ef67fef19e0cb04c2b550cb87fc91db6ff41f7cde51c6119eb3c8 WatchSource:0}: Error finding container 3b147771034ef67fef19e0cb04c2b550cb87fc91db6ff41f7cde51c6119eb3c8: Status 404 returned error can't find the container with id 3b147771034ef67fef19e0cb04c2b550cb87fc91db6ff41f7cde51c6119eb3c8
Jul 29 00:54:32.204928 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.204904    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 00:54:32.239136 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:32.239113    2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-6.ec2.internal\" not found"
Jul 29 00:54:32.315664 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.315640    2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:32.336699 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.336668    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:32.346943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.346924    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 00:54:32.348669 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.348656    2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal"
Jul 29 00:54:32.355974 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.355959    2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 00:54:32.616675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.616599    2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:32.815471 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:32.815270    2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:33.014492 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.014415    2555 apiserver.go:52] "Watching apiserver"
Jul 29 00:54:33.024127 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.023985    2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 00:54:33.024431 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.024400    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-fz8z2","openshift-multus/network-metrics-daemon-rb9df","openshift-network-operator/iptables-alerter-8psd9","kube-system/global-pull-secret-syncer-dsgtn","openshift-cluster-node-tuning-operator/tuned-9z8vp","openshift-dns/node-resolver-w44jh","openshift-image-registry/node-ca-tsn5b","openshift-multus/multus-xlsvm","openshift-network-diagnostics/network-check-target-62pg4","openshift-ovn-kubernetes/ovnkube-node-9sgt7","kube-system/konnectivity-agent-8lx2p","kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal"]
Jul 29 00:54:33.026836 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.026799    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.029086 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.029010    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.029685 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.029661    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6d2n4\""
Jul 29 00:54:33.029776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.029660    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 00:54:33.029776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.029761    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.029884 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.029663    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.031352 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.031331    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.032189 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.032171    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 00:54:33.032293 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.032245    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.032360 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.032331    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.032416 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.032395    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vc797\""
Jul 29 00:54:33.033542 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.033522    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.033632 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.033550    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.033632 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.033610    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:33.034004 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.033979    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k9gmb\""
Jul 29 00:54:33.034083 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.034029    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 00:54:33.034263 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.034242    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.035787 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.035747    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.036523 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.036508    2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 00:54:33.038412 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.038395    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.038513 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.038432    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.038966 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.038950    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9sk8x\""
Jul 29 00:54:33.040241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.039302    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.041512 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.041492    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.042344 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.042326    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sgs2g\""
Jul 29 00:54:33.042868 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.042848    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:33.043316 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.042980    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:33.043316 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.043000    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.044897 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.044872    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-hosts-file\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.044988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.044907    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkm6l\" (UniqueName: \"kubernetes.io/projected/05209071-ba00-4003-8c93-5b47ca7c3d8f-kube-api-access-pkm6l\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.044988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.044933    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30cb1fe8-e263-4406-8fc2-3b269050bc2f-host-slash\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.044988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.044956    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-tmp\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.044988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.044979    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-systemd\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045000    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05209071-ba00-4003-8c93-5b47ca7c3d8f-serviceca\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045020    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-run\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045041    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-lib-modules\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045062    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-socket-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045090    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-etc-selinux\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045115    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.045206 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045149    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysconfig\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045200    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30cb1fe8-e263-4406-8fc2-3b269050bc2f-iptables-alerter-script\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.045549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045255    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be4f8050-958b-471a-8bb9-765583b21e3d-kubelet-config\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.045549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045311    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ln9n\" (UniqueName: \"kubernetes.io/projected/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-kube-api-access-8ln9n\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.045549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045335    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysctl-conf\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045735 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045677    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.045880 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045357    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-var-lib-kubelet\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045950 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045904    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-tuned\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.045950 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045934    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-tmp-dir\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.046049 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045957    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjlm7\" (UniqueName: \"kubernetes.io/projected/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-kube-api-access-bjlm7\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.046049 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.045980    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05209071-ba00-4003-8c93-5b47ca7c3d8f-host\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.046049 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046003    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-device-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.046202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046058    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbt4z\" (UniqueName: \"kubernetes.io/projected/30cb1fe8-e263-4406-8fc2-3b269050bc2f-kube-api-access-pbt4z\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.046202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046094    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be4f8050-958b-471a-8bb9-765583b21e3d-dbus\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.046202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046122    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-kubernetes\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.046202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046148    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-registration-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.046202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046171    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-sys-fs\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.046202 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046193    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-sys\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.046502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046242    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-host\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.046502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046297    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.046502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046332    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-modprobe-d\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.046502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046377    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysctl-d\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.046502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.046411    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnfqp\" (UniqueName: \"kubernetes.io/projected/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-kube-api-access-fnfqp\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.048375 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.048072    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 00:54:33.048375 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.048129    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.048375 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.048188    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:33.048606 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.048585    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 00:54:33.048876 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.048856    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.048939 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.048911    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pthxm\""
Jul 29 00:54:33.049016 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.048863    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.049096 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.049082    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 00:54:33.049946 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.049927    2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 00:54:33.050993 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.050867    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.053081 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.053056    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.053687 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.053657    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 00:54:33.053774 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.053685    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hrdgv\""
Jul 29 00:54:33.053835 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.053812    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 00:54:33.054512 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.054492    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 00:54:33.054665 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.054640    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 00:54:33.054809 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.054764    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 00:54:33.054937 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.054922    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 00:54:33.055563 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.055480    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.055946 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.055931    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 00:54:33.056008 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.055948    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-98hxg\""
Jul 29 00:54:33.056008 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.055987    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 00:54:33.057912 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.057886    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7t24q\""
Jul 29 00:54:33.058142 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.058125    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 00:54:33.074081 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.074024    2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnp7l"
Jul 29 00:54:33.082549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.082205    2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnp7l"
Jul 29 00:54:33.137910 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.137862    2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 00:54:33.146944 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.146902    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.146944 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.146952    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-cni-binary-copy\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.147123 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.146980    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-systemd\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147123 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147009    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05209071-ba00-4003-8c93-5b47ca7c3d8f-serviceca\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.147123 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147033    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-run\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147123 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147056    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-lib-modules\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147123 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147054    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-systemd\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147174    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-run\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147201    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-lib-modules\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147209    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147260    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-etc-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147303    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-env-overrides\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147332    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysconfig\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147357    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-os-release\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.147384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147381    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-kubelet\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147402    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147440    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysconfig\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147456    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147496    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30cb1fe8-e263-4406-8fc2-3b269050bc2f-iptables-alerter-script\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147525    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be4f8050-958b-471a-8bb9-765583b21e3d-kubelet-config\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147534    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/05209071-ba00-4003-8c93-5b47ca7c3d8f-serviceca\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147557    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cnibin\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147583    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-cni-netd\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147605    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-ovnkube-config\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147623    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be4f8050-958b-471a-8bb9-765583b21e3d-kubelet-config\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147650    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ln9n\" (UniqueName: \"kubernetes.io/projected/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-kube-api-access-8ln9n\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147682    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysctl-conf\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.147715 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147705    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-system-cni-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147729    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-daemon-config\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147754    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-run-netns\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147779    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-tmp-dir\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147802    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be4f8050-958b-471a-8bb9-765583b21e3d-dbus\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147827    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-cni-bin\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147836    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysctl-conf\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147850    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-ovn\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147874    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-kubernetes\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147910    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-sys-fs\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147933    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-host\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147968    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.147992    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-etc-kubernetes\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148018    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-modprobe-d\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148066    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnfqp\" (UniqueName: \"kubernetes.io/projected/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-kube-api-access-fnfqp\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148086    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/30cb1fe8-e263-4406-8fc2-3b269050bc2f-iptables-alerter-script\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148091    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-slash\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148116    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-socket-dir-parent\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148145    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-k8s-cni-cncf-io\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148183    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-kubelet\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148209    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148228    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be4f8050-958b-471a-8bb9-765583b21e3d-dbus\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148234    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl48v\" (UniqueName: \"kubernetes.io/projected/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-kube-api-access-vl48v\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148269    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-run-ovn-kubernetes\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148297    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-host\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148313    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-ovnkube-script-lib\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148356    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-kubernetes\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148400    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-sys-fs\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148233    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-modprobe-d\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148448    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-log-socket\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148484    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-cni-bin\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148509    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36158dc8-6ff8-4612-be4a-fccae1ee50f6-konnectivity-ca\") pod \"konnectivity-agent-8lx2p\" (UID: \"36158dc8-6ff8-4612-be4a-fccae1ee50f6\") " pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148550    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-netns\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148559    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-tmp-dir\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.149201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148582    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-socket-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148634    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-etc-selinux\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148678    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148693    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-socket-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148709    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-system-cni-dir\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148733    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36158dc8-6ff8-4612-be4a-fccae1ee50f6-agent-certs\") pod \"konnectivity-agent-8lx2p\" (UID: \"36158dc8-6ff8-4612-be4a-fccae1ee50f6\") " pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148756    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-etc-selinux\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148755    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-cni-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148794    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5v94\" (UniqueName: \"kubernetes.io/projected/9eabb754-06b4-41b5-80cb-bbd954517a16-kube-api-access-d5v94\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.148815    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148816    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-hostroot\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148843    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-multus-certs\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148869    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-var-lib-kubelet\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.148887    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:54:33.648866802 +0000 UTC m=+2.979516401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148910    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-var-lib-kubelet\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148922    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-tuned\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.149930 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148966    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhrjv\" (UniqueName: \"kubernetes.io/projected/034fc553-7c4a-4f22-b467-df36e2e94071-kube-api-access-rhrjv\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.148992    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9eabb754-06b4-41b5-80cb-bbd954517a16-ovn-node-metrics-cert\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149015    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-cni-multus\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149039    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v5kd\" (UniqueName: \"kubernetes.io/projected/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-kube-api-access-2v5kd\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149060    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-systemd-units\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149096    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjlm7\" (UniqueName: \"kubernetes.io/projected/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-kube-api-access-bjlm7\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149189    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05209071-ba00-4003-8c93-5b47ca7c3d8f-host\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149235    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-device-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149260    2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149262    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbt4z\" (UniqueName: \"kubernetes.io/projected/30cb1fe8-e263-4406-8fc2-3b269050bc2f-kube-api-access-pbt4z\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149325    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-cnibin\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149354    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149391    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-registration-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149419    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-sys\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149444    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cni-binary-copy\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149487    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-os-release\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149509    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-conf-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.150549 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149485    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/05209071-ba00-4003-8c93-5b47ca7c3d8f-host\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149534    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149552    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-sys\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149528    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-device-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149590    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysctl-d\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149624    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-registration-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149640    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149682    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-systemd\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149707    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-var-lib-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149739    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-node-log\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149760    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-hosts-file\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149790    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkm6l\" (UniqueName: \"kubernetes.io/projected/05209071-ba00-4003-8c93-5b47ca7c3d8f-kube-api-access-pkm6l\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149815    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-hosts-file\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149812    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-sysctl-d\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149907    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30cb1fe8-e263-4406-8fc2-3b269050bc2f-host-slash\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.149931    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-tmp\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.151138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.150042    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30cb1fe8-e263-4406-8fc2-3b269050bc2f-host-slash\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.152867 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.152792    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-tmp\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.152867 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.152826    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-etc-tuned\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.161384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.161347    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjlm7\" (UniqueName: \"kubernetes.io/projected/cdc09b7a-c22b-4e09-b828-1b7a4ef8c289-kube-api-access-bjlm7\") pod \"node-resolver-w44jh\" (UID: \"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289\") " pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.161671 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.161649    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ln9n\" (UniqueName: \"kubernetes.io/projected/37bced4e-ef99-44e8-b8a1-5e9dd6717abe-kube-api-access-8ln9n\") pod \"aws-ebs-csi-driver-node-rg2rv\" (UID: \"37bced4e-ef99-44e8-b8a1-5e9dd6717abe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.163480 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.163458    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkm6l\" (UniqueName: \"kubernetes.io/projected/05209071-ba00-4003-8c93-5b47ca7c3d8f-kube-api-access-pkm6l\") pod \"node-ca-tsn5b\" (UID: \"05209071-ba00-4003-8c93-5b47ca7c3d8f\") " pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.163589 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.163564    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnfqp\" (UniqueName: \"kubernetes.io/projected/845c72f1-c6fd-4f77-b72e-1e5d04beedcc-kube-api-access-fnfqp\") pod \"tuned-9z8vp\" (UID: \"845c72f1-c6fd-4f77-b72e-1e5d04beedcc\") " pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.165041 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.165016    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbt4z\" (UniqueName: \"kubernetes.io/projected/30cb1fe8-e263-4406-8fc2-3b269050bc2f-kube-api-access-pbt4z\") pod \"iptables-alerter-8psd9\" (UID: \"30cb1fe8-e263-4406-8fc2-3b269050bc2f\") " pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.187138 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.187090    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal" event={"ID":"1c6b9a321efae5b6e2a235b013867e21","Type":"ContainerStarted","Data":"3b147771034ef67fef19e0cb04c2b550cb87fc91db6ff41f7cde51c6119eb3c8"}
Jul 29 00:54:33.188265 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.188233    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal" event={"ID":"df7a3dfe02b1862a6c7a987e8d3facb8","Type":"ContainerStarted","Data":"23fdd926f53da89e53d0f80350d84ebebc5f61ce0422b5f7a9b1521096e7c321"}
Jul 29 00:54:33.250250 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250213    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhrjv\" (UniqueName: \"kubernetes.io/projected/034fc553-7c4a-4f22-b467-df36e2e94071-kube-api-access-rhrjv\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.250250 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250256    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9eabb754-06b4-41b5-80cb-bbd954517a16-ovn-node-metrics-cert\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250307    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-cni-multus\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250329    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v5kd\" (UniqueName: \"kubernetes.io/projected/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-kube-api-access-2v5kd\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250346    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-systemd-units\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250374    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-cnibin\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250399    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250408    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-cni-multus\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250424    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-systemd-units\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250430    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cni-binary-copy\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250472    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-os-release\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250498 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250497    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-conf-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250515    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-cnibin\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250533    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-systemd\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250555    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-var-lib-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250576    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-node-log\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250593    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-conf-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250602    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-systemd\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250640    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-node-log\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250611    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250578    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-os-release\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250689    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-cni-binary-copy\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.250724    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250724    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250754    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-etc-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.250781    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:33.75076183 +0000 UTC m=+3.081411429 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250848    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-env-overrides\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250807    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-etc-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.250959 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250879    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-os-release\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250908    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-kubelet\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250936    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.250659    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-var-lib-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251225    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-openvswitch\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251518    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251566    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cnibin\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251567    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-cni-binary-copy\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251615    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-cni-netd\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251651    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-ovnkube-config\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251682    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-system-cni-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.251728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251714    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-daemon-config\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251745    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-run-netns\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.252323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251782    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-cni-bin\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251807    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-ovn\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.252323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251845    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:33.252323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.251911    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.252323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252319    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252322    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-etc-kubernetes\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252394    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-slash\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252433    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-socket-dir-parent\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252461    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-k8s-cni-cncf-io\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252452    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-etc-kubernetes\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252495    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cni-binary-copy\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252504    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-kubelet\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252540    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252578    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl48v\" (UniqueName: \"kubernetes.io/projected/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-kube-api-access-vl48v\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252594    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-system-cni-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.252603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252346    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-run-ovn\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252632    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-run-ovn-kubernetes\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252671    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-ovnkube-script-lib\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252722    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-kubelet\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252803    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-socket-dir-parent\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252815    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-run-ovn-kubernetes\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252865    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-cnibin\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252883    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-slash\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252918    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-k8s-cni-cncf-io\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252931    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-log-socket\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252969    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-cni-bin\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.252986    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-log-socket\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253022    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36158dc8-6ff8-4612-be4a-fccae1ee50f6-konnectivity-ca\") pod \"konnectivity-agent-8lx2p\" (UID: \"36158dc8-6ff8-4612-be4a-fccae1ee50f6\") " pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253023    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-cni-bin\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253084    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-netns\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253145    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-os-release\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253146    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-system-cni-dir\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253189    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-system-cni-dir\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253211    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-netns\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253187    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36158dc8-6ff8-4612-be4a-fccae1ee50f6-agent-certs\") pod \"konnectivity-agent-8lx2p\" (UID: \"36158dc8-6ff8-4612-be4a-fccae1ee50f6\") " pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253254    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-cni-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253266    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-run-netns\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253296    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5v94\" (UniqueName: \"kubernetes.io/projected/9eabb754-06b4-41b5-80cb-bbd954517a16-kube-api-access-d5v94\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253315    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-cni-dir\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253331    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-cni-netd\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253360    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-hostroot\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253391    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-multus-certs\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253433    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253518    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-var-lib-cni-bin\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.253776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.253720    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-ovnkube-script-lib\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.254442 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.254177    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-kubelet\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.254442 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.254228    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-host-run-multus-certs\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.254442 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.254346    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-ovnkube-config\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.254442 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.254395    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36158dc8-6ff8-4612-be4a-fccae1ee50f6-konnectivity-ca\") pod \"konnectivity-agent-8lx2p\" (UID: \"36158dc8-6ff8-4612-be4a-fccae1ee50f6\") " pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.254617 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.254548    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9eabb754-06b4-41b5-80cb-bbd954517a16-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.254756 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.254731    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-multus-daemon-config\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.255376 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.255344    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9eabb754-06b4-41b5-80cb-bbd954517a16-env-overrides\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.255465 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.255426    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-hostroot\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.255465 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.255458    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9eabb754-06b4-41b5-80cb-bbd954517a16-ovn-node-metrics-cert\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.257346 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.257325    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36158dc8-6ff8-4612-be4a-fccae1ee50f6-agent-certs\") pod \"konnectivity-agent-8lx2p\" (UID: \"36158dc8-6ff8-4612-be4a-fccae1ee50f6\") " pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.258306 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.258270    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:33.258306 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.258309    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:33.258440 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.258323    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:33.258440 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.258377    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:33.758360486 +0000 UTC m=+3.089010087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:33.259938 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.259916    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v5kd\" (UniqueName: \"kubernetes.io/projected/2ecae3a5-f00e-445d-b790-0da7f43e9e8c-kube-api-access-2v5kd\") pod \"multus-xlsvm\" (UID: \"2ecae3a5-f00e-445d-b790-0da7f43e9e8c\") " pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.260458 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.260426    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhrjv\" (UniqueName: \"kubernetes.io/projected/034fc553-7c4a-4f22-b467-df36e2e94071-kube-api-access-rhrjv\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.261870 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.261847    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl48v\" (UniqueName: \"kubernetes.io/projected/5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456-kube-api-access-vl48v\") pod \"multus-additional-cni-plugins-fz8z2\" (UID: \"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456\") " pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.262354 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.262331    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5v94\" (UniqueName: \"kubernetes.io/projected/9eabb754-06b4-41b5-80cb-bbd954517a16-kube-api-access-d5v94\") pod \"ovnkube-node-9sgt7\" (UID: \"9eabb754-06b4-41b5-80cb-bbd954517a16\") " pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.340035 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.339944    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tsn5b"
Jul 29 00:54:33.355030 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.355001    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv"
Jul 29 00:54:33.365731 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.365706    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8psd9"
Jul 29 00:54:33.370492 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.370470    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9z8vp"
Jul 29 00:54:33.378135 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.378107    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w44jh"
Jul 29 00:54:33.385757 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.385733    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fz8z2"
Jul 29 00:54:33.392369 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.392354    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:33.399220 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.399196    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:33.405372 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.405356    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xlsvm"
Jul 29 00:54:33.413552 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.413532    2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 00:54:33.656066 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.655971    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:33.656227 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.656097    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:33.656227 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.656171    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:54:34.65614489 +0000 UTC m=+3.986794478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:33.757089 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.757052    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:33.757226 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.757173    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:33.757289 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.757227    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:34.757213065 +0000 UTC m=+4.087862649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:33.857538 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:33.857501    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:33.857716 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.857694    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:33.857764 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.857725    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:33.857764 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.857738    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:33.857840 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:33.857807    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:34.857787487 +0000 UTC m=+4.188437073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:34.057619 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.057586    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36158dc8_6ff8_4612_be4a_fccae1ee50f6.slice/crio-9e8650943825a350ea217d1fe7569edbc40ea37a839bd6fa79204a310044dabf WatchSource:0}: Error finding container 9e8650943825a350ea217d1fe7569edbc40ea37a839bd6fa79204a310044dabf: Status 404 returned error can't find the container with id 9e8650943825a350ea217d1fe7569edbc40ea37a839bd6fa79204a310044dabf
Jul 29 00:54:34.059400 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.059380    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30cb1fe8_e263_4406_8fc2_3b269050bc2f.slice/crio-a2cc18be4c76897c9b9f114eb33108e041e5816250f23c48658f76dc674ec628 WatchSource:0}: Error finding container a2cc18be4c76897c9b9f114eb33108e041e5816250f23c48658f76dc674ec628: Status 404 returned error can't find the container with id a2cc18be4c76897c9b9f114eb33108e041e5816250f23c48658f76dc674ec628
Jul 29 00:54:34.060041 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.060019    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc09b7a_c22b_4e09_b828_1b7a4ef8c289.slice/crio-7be244914acb360992338837a21f648445605f1a12afa70c96f06b100b5639b6 WatchSource:0}: Error finding container 7be244914acb360992338837a21f648445605f1a12afa70c96f06b100b5639b6: Status 404 returned error can't find the container with id 7be244914acb360992338837a21f648445605f1a12afa70c96f06b100b5639b6
Jul 29 00:54:34.060699 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.060649    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd7e544_ea04_43b9_8f8d_c9bb9e7ec456.slice/crio-4b38bbae50c0397d77f7f8d68dde198214d504c1c8a751397c2f0a3b7d7d62b6 WatchSource:0}: Error finding container 4b38bbae50c0397d77f7f8d68dde198214d504c1c8a751397c2f0a3b7d7d62b6: Status 404 returned error can't find the container with id 4b38bbae50c0397d77f7f8d68dde198214d504c1c8a751397c2f0a3b7d7d62b6
Jul 29 00:54:34.062231 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.062054    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05209071_ba00_4003_8c93_5b47ca7c3d8f.slice/crio-8f142afc42b3759bc847e12aa9e001855c8e88480650a254f4725ac93fe3d372 WatchSource:0}: Error finding container 8f142afc42b3759bc847e12aa9e001855c8e88480650a254f4725ac93fe3d372: Status 404 returned error can't find the container with id 8f142afc42b3759bc847e12aa9e001855c8e88480650a254f4725ac93fe3d372
Jul 29 00:54:34.063627 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.063606    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eabb754_06b4_41b5_80cb_bbd954517a16.slice/crio-b0123cb1cc1463d946f81e7a8b01da6c77644f947419a44e24e024644845cbda WatchSource:0}: Error finding container b0123cb1cc1463d946f81e7a8b01da6c77644f947419a44e24e024644845cbda: Status 404 returned error can't find the container with id b0123cb1cc1463d946f81e7a8b01da6c77644f947419a44e24e024644845cbda
Jul 29 00:54:34.067382 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.067152    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ecae3a5_f00e_445d_b790_0da7f43e9e8c.slice/crio-751c3669d47a29d8bfb53543a868518e687acfc5a154de62bde410fc3faf6acc WatchSource:0}: Error finding container 751c3669d47a29d8bfb53543a868518e687acfc5a154de62bde410fc3faf6acc: Status 404 returned error can't find the container with id 751c3669d47a29d8bfb53543a868518e687acfc5a154de62bde410fc3faf6acc
Jul 29 00:54:34.085028 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.084979    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:49:33 +0000 UTC" deadline="2028-05-09 12:12:52.142155372 +0000 UTC"
Jul 29 00:54:34.085028 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.085020    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15611h18m18.057138459s"
Jul 29 00:54:34.088869 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:54:34.088852    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bced4e_ef99_44e8_b8a1_5e9dd6717abe.slice/crio-a8b213cbd03bfc48780474ec30b0cfb696ae59e7a179e27693f89e3ff197ebb0 WatchSource:0}: Error finding container a8b213cbd03bfc48780474ec30b0cfb696ae59e7a179e27693f89e3ff197ebb0: Status 404 returned error can't find the container with id a8b213cbd03bfc48780474ec30b0cfb696ae59e7a179e27693f89e3ff197ebb0
Jul 29 00:54:34.184148 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.184124    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:34.184292 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.184246    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:34.192251 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.192224    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal" event={"ID":"df7a3dfe02b1862a6c7a987e8d3facb8","Type":"ContainerStarted","Data":"dadb662b3a5d96ea99b7c520b4a967b416e2fb6c0986f7159a98ba2fb1beb7d2"}
Jul 29 00:54:34.193367 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.193333    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv" event={"ID":"37bced4e-ef99-44e8-b8a1-5e9dd6717abe","Type":"ContainerStarted","Data":"a8b213cbd03bfc48780474ec30b0cfb696ae59e7a179e27693f89e3ff197ebb0"}
Jul 29 00:54:34.194968 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.194948    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xlsvm" event={"ID":"2ecae3a5-f00e-445d-b790-0da7f43e9e8c","Type":"ContainerStarted","Data":"751c3669d47a29d8bfb53543a868518e687acfc5a154de62bde410fc3faf6acc"}
Jul 29 00:54:34.196006 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.195983    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerStarted","Data":"4b38bbae50c0397d77f7f8d68dde198214d504c1c8a751397c2f0a3b7d7d62b6"}
Jul 29 00:54:34.196943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.196922    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w44jh" event={"ID":"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289","Type":"ContainerStarted","Data":"7be244914acb360992338837a21f648445605f1a12afa70c96f06b100b5639b6"}
Jul 29 00:54:34.198271 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.198229    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8psd9" event={"ID":"30cb1fe8-e263-4406-8fc2-3b269050bc2f","Type":"ContainerStarted","Data":"a2cc18be4c76897c9b9f114eb33108e041e5816250f23c48658f76dc674ec628"}
Jul 29 00:54:34.199327 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.199306    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9z8vp" event={"ID":"845c72f1-c6fd-4f77-b72e-1e5d04beedcc","Type":"ContainerStarted","Data":"5e68244eca3605b4d671e48a17d20148c6d5e5ca5ecf7d78324fc47de19d91a2"}
Jul 29 00:54:34.200488 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.200460    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"b0123cb1cc1463d946f81e7a8b01da6c77644f947419a44e24e024644845cbda"}
Jul 29 00:54:34.201721 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.201702    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tsn5b" event={"ID":"05209071-ba00-4003-8c93-5b47ca7c3d8f","Type":"ContainerStarted","Data":"8f142afc42b3759bc847e12aa9e001855c8e88480650a254f4725ac93fe3d372"}
Jul 29 00:54:34.202826 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.202803    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8lx2p" event={"ID":"36158dc8-6ff8-4612-be4a-fccae1ee50f6","Type":"ContainerStarted","Data":"9e8650943825a350ea217d1fe7569edbc40ea37a839bd6fa79204a310044dabf"}
Jul 29 00:54:34.208334 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.208242    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-6.ec2.internal" podStartSLOduration=2.208226434 podStartE2EDuration="2.208226434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:34.207975562 +0000 UTC m=+3.538625167" watchObservedRunningTime="2026-07-29 00:54:34.208226434 +0000 UTC m=+3.538876047"
Jul 29 00:54:34.664993 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.664918    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:34.665162 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.665089    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:34.665225 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.665174    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:54:36.665153539 +0000 UTC m=+5.995803132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:34.765952 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.765916    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:34.766110 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.766072    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:34.766184 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.766135    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:36.766116291 +0000 UTC m=+6.096765883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:34.867320 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:34.867168    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:34.867501 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.867359    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:34.867501 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.867396    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:34.867501 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.867409    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:34.867501 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:34.867467    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:36.867449302 +0000 UTC m=+6.198098888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:35.085628 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:35.085537    2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 00:49:33 +0000 UTC" deadline="2028-04-19 17:17:11.141503262 +0000 UTC"
Jul 29 00:54:35.085628 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:35.085577    2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15136h22m36.055931357s"
Jul 29 00:54:35.186404 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:35.185689    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:35.186404 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:35.185814    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:35.186404 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:35.186218    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:35.186404 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:35.186336    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:35.220349 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:35.219514    2555 generic.go:356] "Generic (PLEG): container finished" podID="1c6b9a321efae5b6e2a235b013867e21" containerID="29f399baabc6958cc0acb4eebd7b9502c507040029ef924de1cbb7a110adad22" exitCode=0
Jul 29 00:54:35.220349 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:35.220058    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal" event={"ID":"1c6b9a321efae5b6e2a235b013867e21","Type":"ContainerDied","Data":"29f399baabc6958cc0acb4eebd7b9502c507040029ef924de1cbb7a110adad22"}
Jul 29 00:54:36.184103 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:36.183627    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:36.184103 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.183755    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:36.238755 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:36.238717    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal" event={"ID":"1c6b9a321efae5b6e2a235b013867e21","Type":"ContainerStarted","Data":"9f07fd74ea8d8b96487b0e8cb954fb7f5cb72d8f403d582a495062dceed0767b"}
Jul 29 00:54:36.684201 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:36.684163    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:36.684391 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.684347    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:36.684465 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.684440    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:54:40.684409019 +0000 UTC m=+10.015058622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:36.785434 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:36.785395    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:36.785632 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.785549    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:36.785632 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.785612    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:40.785594119 +0000 UTC m=+10.116243706 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:36.886377 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:36.886331    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:36.886559 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.886495    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:36.886559 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.886519    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:36.886559 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.886531    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:36.886724 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:36.886594    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:40.886573439 +0000 UTC m=+10.217223025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:37.185868 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:37.185838    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:37.186340 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:37.185979    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:37.186400 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:37.186360    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:37.186486 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:37.186462    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:38.183943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:38.183898    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:38.184133 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:38.184077    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:39.185131 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:39.185098    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:39.185588 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:39.185220    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:39.185588 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:39.185392    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:39.185588 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:39.185511    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:40.183399 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:40.183360    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:40.183573 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.183488    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:40.717071 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:40.716977    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:40.717571 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.717145    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:40.717571 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.717212    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:54:48.717192091 +0000 UTC m=+18.047841692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:40.818471 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:40.818389    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:40.818659 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.818529    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:40.818659 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.818598    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:48.818578964 +0000 UTC m=+18.149228547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:40.919120 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:40.919083    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:40.919310 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.919245    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:40.919310 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.919264    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:40.919310 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.919292    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:40.919456 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:40.919356    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:54:48.91933739 +0000 UTC m=+18.249986975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:41.184731 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:41.184660    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:41.184923 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:41.184751    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:41.184923 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:41.184776    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:41.184923 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:41.184878    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:42.184038 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:42.183535    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:42.184038 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:42.183661    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:43.183400 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:43.183360    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:43.183400 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:43.183385    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:43.183615 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:43.183508    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:43.183615 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:43.183600    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:44.183457 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:44.183423    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:44.183932 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:44.183540    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:45.184226 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:45.184139    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:45.184636 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:45.184151    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:45.184636 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:45.184289    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:45.184636 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:45.184368    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:46.184043 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:46.184015    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:46.184204 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:46.184117    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:47.184158 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:47.184118    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:47.184591 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:47.184256    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:47.184591 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:47.184307    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:47.184591 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:47.184407    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:48.183880 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:48.183845    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:48.184106 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.183948    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:48.783136 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:48.783104    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:48.783583 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.783221    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:48.783583 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.783269    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:55:04.783256471 +0000 UTC m=+34.113906056 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:54:48.883459 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:48.883422    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:48.883640 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.883609    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:48.883742 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.883691    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:04.883669687 +0000 UTC m=+34.214319286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:54:48.984635 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:48.984602    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:48.984806 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.984742    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:54:48.984806 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.984768    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:54:48.984806 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.984781    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:48.984952 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:48.984846    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:04.984826997 +0000 UTC m=+34.315476615 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:54:49.183655 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:49.183570    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:49.183816 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:49.183572    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:49.183816 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:49.183717    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:49.183816 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:49.183771    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:50.183207 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:50.183153    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:50.183673 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:50.183299    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:51.185010 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:51.184977    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:51.185010 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:51.184995    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:51.185477 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:51.185101    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:51.185477 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:51.185211    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:52.183928 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:52.183893    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:52.184099 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:52.184027    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:53.184246 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.183861    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:53.184246 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:53.184197    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:53.184246 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.183908    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:53.184838 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:53.184324    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:53.270782 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.270672    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv" event={"ID":"37bced4e-ef99-44e8-b8a1-5e9dd6717abe","Type":"ContainerStarted","Data":"2f14fc7018b5db1b09cfc1b5ad8070dafc21d6c2c2016816debccec49cda5f79"}
Jul 29 00:54:53.272004 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.271981    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456" containerID="225b69b9ded0ff418420f887cdab76ae280be99012a14a9bf691f119dc1dea61" exitCode=0
Jul 29 00:54:53.272084 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.272041    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerDied","Data":"225b69b9ded0ff418420f887cdab76ae280be99012a14a9bf691f119dc1dea61"}
Jul 29 00:54:53.274342 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.274318    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w44jh" event={"ID":"cdc09b7a-c22b-4e09-b828-1b7a4ef8c289","Type":"ContainerStarted","Data":"589b38bb00c776679365963f97aebedcc2344569f7f3c1a5ed2b6e7adca61239"}
Jul 29 00:54:53.276247 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.276170    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9z8vp" event={"ID":"845c72f1-c6fd-4f77-b72e-1e5d04beedcc","Type":"ContainerStarted","Data":"3b845c59a8e2d9e3c36655c1b505f0dd7965039cf6904d4912d0c5b3c6af82b8"}
Jul 29 00:54:53.279076 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.279052    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"54ad1ae67a959e6aa321b4360d70998309fd691fff6a1db903b96b31e0d1bc82"}
Jul 29 00:54:53.279155 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.279133    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"3909dba642372d2563590539acd267e1475a132531c7a16a07ea6fd86a6cf47a"}
Jul 29 00:54:53.280940 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.280846    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tsn5b" event={"ID":"05209071-ba00-4003-8c93-5b47ca7c3d8f","Type":"ContainerStarted","Data":"dcb201f36f2f07de1fc942c48de2447590b6fcc914002154346bc04f10fd529a"}
Jul 29 00:54:53.282244 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.282225    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8lx2p" event={"ID":"36158dc8-6ff8-4612-be4a-fccae1ee50f6","Type":"ContainerStarted","Data":"503dbc01d4380a843bb1f2622ea2819a07029feb5d8fbc5e6750d04e615818c4"}
Jul 29 00:54:53.296241 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.296194    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-6.ec2.internal" podStartSLOduration=21.29617614 podStartE2EDuration="21.29617614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:36.256530546 +0000 UTC m=+5.587180153" watchObservedRunningTime="2026-07-29 00:54:53.29617614 +0000 UTC m=+22.626825748"
Jul 29 00:54:53.309045 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.308996    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8lx2p" podStartSLOduration=7.554882244 podStartE2EDuration="22.308978241s" totalImagesPullingTime="14.754095997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:53.308967603 +0000 UTC m=+22.639617220" watchObservedRunningTime="2026-07-29 00:54:53.308978241 +0000 UTC m=+22.639627849"
Jul 29 00:54:53.322017 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.321978    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tsn5b" podStartSLOduration=3.479242917 podStartE2EDuration="22.321963466s" totalImagesPullingTime="18.842720549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:53.321582997 +0000 UTC m=+22.652232602" watchObservedRunningTime="2026-07-29 00:54:53.321963466 +0000 UTC m=+22.652613071"
Jul 29 00:54:53.336777 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.336738    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9z8vp" podStartSLOduration=3.516000803 podStartE2EDuration="22.33672455s" totalImagesPullingTime="18.820723747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:53.336522557 +0000 UTC m=+22.667172187" watchObservedRunningTime="2026-07-29 00:54:53.33672455 +0000 UTC m=+22.667374156"
Jul 29 00:54:53.350394 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.350352    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w44jh" podStartSLOduration=3.506592804 podStartE2EDuration="22.350339068s" totalImagesPullingTime="18.843746264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:53.350243265 +0000 UTC m=+22.680892872" watchObservedRunningTime="2026-07-29 00:54:53.350339068 +0000 UTC m=+22.680988674"
Jul 29 00:54:53.408126 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.408093    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:53.471655 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:53.471626    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:54.183159 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.183134    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:54.183298 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:54.183233    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:54.286983 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.286950    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xlsvm" event={"ID":"2ecae3a5-f00e-445d-b790-0da7f43e9e8c","Type":"ContainerStarted","Data":"b99220dc5dcdeae890b420f0a7b2f41417d9b335ea0c9e0d1b0902870687a805"}
Jul 29 00:54:54.288885 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.288816    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8psd9" event={"ID":"30cb1fe8-e263-4406-8fc2-3b269050bc2f","Type":"ContainerStarted","Data":"4a737080c0b4ccaf13c82623cb6c60d1f11c6b0be98448f72a63265b2a0ce5b9"}
Jul 29 00:54:54.292616 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.292590    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"1305ed15dd1c6701aaa9984170a1cfc52c86ff7947e79769484d0edf5a5d312a"}
Jul 29 00:54:54.292712 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.292673    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"d778ea052e7e267cd14727e1a7fb92ed18b81c4472f3ca300c3fa296abea43a5"}
Jul 29 00:54:54.292754 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.292729    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"674aeabdd1adfd5bc523a64f8e08f8749c007fa2e4b970264b1f4a4d3a082cf1"}
Jul 29 00:54:54.292803 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.292778    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"1e2a3956b30b79df25cf96bff9e3931d55da4894f4087fd2636cbda1392750cc"}
Jul 29 00:54:54.303908 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.303870    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xlsvm" podStartSLOduration=4.205915943 podStartE2EDuration="23.303853698s" totalImagesPullingTime="19.097937755s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:54.30367561 +0000 UTC m=+23.634325228" watchObservedRunningTime="2026-07-29 00:54:54.303853698 +0000 UTC m=+23.634503304"
Jul 29 00:54:54.316952 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.316917    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8psd9" podStartSLOduration=4.473167627 podStartE2EDuration="23.316900756s" totalImagesPullingTime="18.843733129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:54.316469647 +0000 UTC m=+23.647119254" watchObservedRunningTime="2026-07-29 00:54:54.316900756 +0000 UTC m=+23.647550363"
Jul 29 00:54:54.363701 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.363677    2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 00:54:54.442868 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.442833    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:54.544031 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.543992    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:54.644932 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:54.644906    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:55.120367 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.120332    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:55.128243 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.128143    2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T00:54:54.36369898Z","UUID":"2cca1843-3024-4086-ba38-d8dd79934bcf","Handler":null,"Name":"","Endpoint":""}
Jul 29 00:54:55.129894 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.129845    2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 00:54:55.129894 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.129885    2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 00:54:55.183680 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.183651    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:55.183842 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.183657    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:55.183842 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:55.183784    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:55.183967 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:55.183862    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:55.220519 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.220487    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:55.296725 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.296685    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv" event={"ID":"37bced4e-ef99-44e8-b8a1-5e9dd6717abe","Type":"ContainerStarted","Data":"4bdd8ac57937e93ad8e1483680c57fc592bac7f8753b9a96420481513dca1468"}
Jul 29 00:54:55.316309 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.316269    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:55.316932 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:55.316914    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:56.183892 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.183667    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:56.184046 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:56.183972    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:56.301686 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.301640    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv" event={"ID":"37bced4e-ef99-44e8-b8a1-5e9dd6717abe","Type":"ContainerStarted","Data":"f56e395a125be4b7e9f488049b55a6366a4762c60aae44ba8fbdb8c776fbe99a"}
Jul 29 00:54:56.305707 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.305671    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"7c854610118317cbe5f42499bc2bb825170373ca7834c9a9eff252aac06eeaaa"}
Jul 29 00:54:56.305839 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.305721    2555 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 00:54:56.319250 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.319209    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rg2rv" podStartSLOduration=4.451735005 podStartE2EDuration="25.319196176s" totalImagesPullingTime="20.867461171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:56.318680943 +0000 UTC m=+25.649330545" watchObservedRunningTime="2026-07-29 00:54:56.319196176 +0000 UTC m=+25.649845781"
Jul 29 00:54:56.981062 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.981033    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:56.981681 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:56.981656    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8lx2p"
Jul 29 00:54:57.036479 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:57.036441    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:54:57.183873 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:57.183843    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:57.184041 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:57.183949    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:57.184041 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:57.183990    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:57.184158 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:57.184059    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:58.183990 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.183815    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:58.184630 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:58.184054    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:54:58.315989 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.315904    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" event={"ID":"9eabb754-06b4-41b5-80cb-bbd954517a16","Type":"ContainerStarted","Data":"93b4741c22866fc92e4753b9de12f7e793da74dd33cbdca7810d519793f0ed31"}
Jul 29 00:54:58.316163 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.316134    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:58.317634 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.317604    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456" containerID="709690a0e9eb8ec8c1b01f096c47137776ba92d439b1117c6c19ada80ba84798" exitCode=0
Jul 29 00:54:58.317749 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.317686    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerDied","Data":"709690a0e9eb8ec8c1b01f096c47137776ba92d439b1117c6c19ada80ba84798"}
Jul 29 00:54:58.331772 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.331750    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:58.345717 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:58.344386    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7" podStartSLOduration=8.481616582000001 podStartE2EDuration="27.34436665s" totalImagesPullingTime="18.862750068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:54:58.341549745 +0000 UTC m=+27.672199351" watchObservedRunningTime="2026-07-29 00:54:58.34436665 +0000 UTC m=+27.675016259"
Jul 29 00:54:59.186335 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.185761    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:59.186335 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:59.185891    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:59.186335 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.186027    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:59.186335 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:59.186140    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:59.320401 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.320371    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:59.320557 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.320452    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:59.335149 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.335126    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:54:59.905256 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.905023    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dsgtn"]
Jul 29 00:54:59.905407 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.905299    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:54:59.905465 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:59.905413    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:54:59.908290 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.908246    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rb9df"]
Jul 29 00:54:59.908414 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.908383    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:54:59.908496 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:59.908479    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:54:59.908971 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.908953    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62pg4"]
Jul 29 00:54:59.909052 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:54:59.909040    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:54:59.909146 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:54:59.909126    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:55:00.324506 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:00.324480    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456" containerID="0b8de912fa1adb6241226fcd56b06bb1aa57f5797a27b7f87ddddc5e9c443fd2" exitCode=0
Jul 29 00:55:00.324866 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:00.324563    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerDied","Data":"0b8de912fa1adb6241226fcd56b06bb1aa57f5797a27b7f87ddddc5e9c443fd2"}
Jul 29 00:55:00.920004 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:00.919971    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:00.969616 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:00.969585    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:01.184786 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:01.184712    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:01.184969 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:01.184812    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:55:01.184969 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:01.184859    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:01.184969 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:01.184944    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:55:02.184039 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:02.183954    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:55:02.184594 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:02.184058    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:55:02.330689 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:02.330659    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456" containerID="6c424cf4360daaa1b4fb59b9ae8b5dd3c5841842c80ec0b76d515b6597e3abf3" exitCode=0
Jul 29 00:55:02.330845 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:02.330715    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerDied","Data":"6c424cf4360daaa1b4fb59b9ae8b5dd3c5841842c80ec0b76d515b6597e3abf3"}
Jul 29 00:55:03.184298 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:03.184201    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:03.184857 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:03.184213    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:03.184857 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:03.184358    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:55:03.184857 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:03.184389    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:55:03.914228 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:03.914197    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:03.965548 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:03.965515    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:04.183852 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:04.183773    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:55:04.183995 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:04.183892    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:55:04.567040 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:04.567003    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:04.674894 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:04.674860    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:04.804321 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:04.804288    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:04.804475 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:04.804431    2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 00:55:04.804522 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:04.804498    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret podName:be4f8050-958b-471a-8bb9-765583b21e3d nodeName:}" failed. No retries permitted until 2026-07-29 00:55:36.804480842 +0000 UTC m=+66.135130452 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret") pod "global-pull-secret-syncer-dsgtn" (UID: "be4f8050-958b-471a-8bb9-765583b21e3d") : object "kube-system"/"original-pull-secret" not registered
Jul 29 00:55:04.904907 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:04.904816    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:55:04.905051 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:04.904969    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:55:04.905051 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:04.905036    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:36.905019367 +0000 UTC m=+66.235668952 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 00:55:05.005248 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:05.005213    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:05.005433 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:05.005359    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 00:55:05.005433 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:05.005380    2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 00:55:05.005433 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:05.005392    2555 projected.go:196] Error preparing data for projected volume kube-api-access-h8mc6 for pod openshift-network-diagnostics/network-check-target-62pg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:55:05.005554 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:05.005446    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6 podName:bbea195e-3ebe-4216-8d89-8963e218f2d5 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:37.00543277 +0000 UTC m=+66.336082367 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h8mc6" (UniqueName: "kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6") pod "network-check-target-62pg4" (UID: "bbea195e-3ebe-4216-8d89-8963e218f2d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 00:55:05.189146 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:05.189117    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:05.189418 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:05.189126    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:05.189418 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:05.189219    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62pg4" podUID="bbea195e-3ebe-4216-8d89-8963e218f2d5"
Jul 29 00:55:05.189418 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:05.189338    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dsgtn" podUID="be4f8050-958b-471a-8bb9-765583b21e3d"
Jul 29 00:55:05.978546 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:05.978505    2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-6.ec2.internal" event="NodeReady"
Jul 29 00:55:05.979018 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:05.978650    2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 00:55:06.019905 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.019870    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hpq5j"]
Jul 29 00:55:06.028881 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.028854    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k2bfw"]
Jul 29 00:55:06.029045 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.029000    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.031614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.031593    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9mcp8\""
Jul 29 00:55:06.031614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.031606    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 00:55:06.031825 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.031627    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 00:55:06.034725 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.034693    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.034965 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.034913    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hpq5j"]
Jul 29 00:55:06.037463 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.037441    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 00:55:06.037463 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.037456    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 00:55:06.037602 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.037500    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 00:55:06.037752 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.037735    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-58969\""
Jul 29 00:55:06.046997 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.046973    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k2bfw"]
Jul 29 00:55:06.132174 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.132130    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:06.183200 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.183171    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:55:06.183551 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.183513    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:06.185821 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.185797    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 00:55:06.185941 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.185921    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bxdbf\""
Jul 29 00:55:06.213844 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.213812    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b249760f-1f36-4430-ab58-22f44d91f7a9-cert\") pod \"ingress-canary-k2bfw\" (UID: \"b249760f-1f36-4430-ab58-22f44d91f7a9\") " pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.213988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.213854    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-tmp-dir\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.213988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.213887    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-672z2\" (UniqueName: \"kubernetes.io/projected/b249760f-1f36-4430-ab58-22f44d91f7a9-kube-api-access-672z2\") pod \"ingress-canary-k2bfw\" (UID: \"b249760f-1f36-4430-ab58-22f44d91f7a9\") " pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.213988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.213941    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.213988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.213960    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-config-volume\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.213988 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.213976    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7rrv\" (UniqueName: \"kubernetes.io/projected/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-kube-api-access-q7rrv\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.315282 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315185    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.315282 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315235    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-config-volume\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.315517 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315267    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7rrv\" (UniqueName: \"kubernetes.io/projected/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-kube-api-access-q7rrv\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.315517 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:06.315352    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:06.315517 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:06.315426    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:06.815409199 +0000 UTC m=+36.146058782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:06.315517 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315493    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b249760f-1f36-4430-ab58-22f44d91f7a9-cert\") pod \"ingress-canary-k2bfw\" (UID: \"b249760f-1f36-4430-ab58-22f44d91f7a9\") " pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.315737 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315520    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-tmp-dir\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.315737 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315542    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-672z2\" (UniqueName: \"kubernetes.io/projected/b249760f-1f36-4430-ab58-22f44d91f7a9-kube-api-access-672z2\") pod \"ingress-canary-k2bfw\" (UID: \"b249760f-1f36-4430-ab58-22f44d91f7a9\") " pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.315903 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315873    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-tmp-dir\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.316016 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.315978    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-config-volume\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.319901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.319882    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b249760f-1f36-4430-ab58-22f44d91f7a9-cert\") pod \"ingress-canary-k2bfw\" (UID: \"b249760f-1f36-4430-ab58-22f44d91f7a9\") " pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.324771 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.324752    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7rrv\" (UniqueName: \"kubernetes.io/projected/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-kube-api-access-q7rrv\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.325249 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.325228    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-672z2\" (UniqueName: \"kubernetes.io/projected/b249760f-1f36-4430-ab58-22f44d91f7a9-kube-api-access-672z2\") pod \"ingress-canary-k2bfw\" (UID: \"b249760f-1f36-4430-ab58-22f44d91f7a9\") " pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.348384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.348356    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k2bfw"
Jul 29 00:55:06.819664 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:06.819626    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:06.819840 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:06.819791    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:06.819898 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:06.819877    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:07.819854714 +0000 UTC m=+37.150504298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:07.186292 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.186253    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:07.186705 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.186453    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:07.189147 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.189130    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 00:55:07.190374 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.190357    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 00:55:07.190460 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.190378    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ndg5h\""
Jul 29 00:55:07.190460 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.190357    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 00:55:07.278599 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.278563    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k2bfw"]
Jul 29 00:55:07.337556 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:55:07.337529    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb249760f_1f36_4430_ab58_22f44d91f7a9.slice/crio-ddc0f591c4b2537bfc26aa617b11c568d8050cc56da7b321d92f9f12453286fe WatchSource:0}: Error finding container ddc0f591c4b2537bfc26aa617b11c568d8050cc56da7b321d92f9f12453286fe: Status 404 returned error can't find the container with id ddc0f591c4b2537bfc26aa617b11c568d8050cc56da7b321d92f9f12453286fe
Jul 29 00:55:07.343383 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.343353    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k2bfw" event={"ID":"b249760f-1f36-4430-ab58-22f44d91f7a9","Type":"ContainerStarted","Data":"ddc0f591c4b2537bfc26aa617b11c568d8050cc56da7b321d92f9f12453286fe"}
Jul 29 00:55:07.826358 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:07.826335    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:07.826501 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:07.826452    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:07.826541 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:07.826505    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:09.826493103 +0000 UTC m=+39.157142687 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:08.349110 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:08.349067    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456" containerID="1265b749b1767f318ccae4624d1a478b36eb07d4b8a3034540260a6aec47d299" exitCode=0
Jul 29 00:55:08.349610 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:08.349132    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerDied","Data":"1265b749b1767f318ccae4624d1a478b36eb07d4b8a3034540260a6aec47d299"}
Jul 29 00:55:09.355050 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:09.354789    2555 generic.go:356] "Generic (PLEG): container finished" podID="5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456" containerID="274693d0e1a6888b866962f39245b751bdbe78bff973157e9fd4549afb1daf17" exitCode=0
Jul 29 00:55:09.355050 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:09.354872    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerDied","Data":"274693d0e1a6888b866962f39245b751bdbe78bff973157e9fd4549afb1daf17"}
Jul 29 00:55:09.843400 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:09.843366    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:09.843588 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:09.843539    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:09.843654 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:09.843620    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:13.843596913 +0000 UTC m=+43.174246497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:10.360330 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:10.360224    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" event={"ID":"5bd7e544-ea04-43b9-8f8d-c9bb9e7ec456","Type":"ContainerStarted","Data":"61519e21a7290a44d69d26e5d2ad8fa5895d0550ebdad8d457675ca875c17f38"}
Jul 29 00:55:10.361686 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:10.361660    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k2bfw" event={"ID":"b249760f-1f36-4430-ab58-22f44d91f7a9","Type":"ContainerStarted","Data":"1cc76c142db64bc81644a29759a3791dfce7abe327beaed273ba21a3710081ac"}
Jul 29 00:55:10.382806 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:10.382761    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fz8z2" podStartSLOduration=8.970299675 podStartE2EDuration="39.382744105s" totalImagesPullingTime="30.41244443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:10.380747058 +0000 UTC m=+39.711396664" watchObservedRunningTime="2026-07-29 00:55:10.382744105 +0000 UTC m=+39.713393714"
Jul 29 00:55:10.397792 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:10.397755    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k2bfw" podStartSLOduration=1.691734253 podStartE2EDuration="4.397741985s" totalImagesPullingTime="2.706007732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:10.397505095 +0000 UTC m=+39.728154714" watchObservedRunningTime="2026-07-29 00:55:10.397741985 +0000 UTC m=+39.728391592"
Jul 29 00:55:10.482290 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:10.482249    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:10.538702 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:10.538678    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:11.269097 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:11.269060    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:11.320379 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:11.320342    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:11.777737 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:11.777709    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:11.877746 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:11.877718    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:13.872163 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:13.872124    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:13.872545 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:13.872243    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:13.872545 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:13.872311    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:21.872294842 +0000 UTC m=+51.202944428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:21.929129 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:21.929096    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:21.929576 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:21.929193    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:21.929576 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:21.929244    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:55:37.929230806 +0000 UTC m=+67.259880390 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:25.914869 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:25.914836    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:26.012583 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:26.012549    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:31.343013 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:31.342987    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9sgt7"
Jul 29 00:55:31.447657 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:31.447621    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:32.275266 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:32.275230    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:32.332612 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:32.332585    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:55:36.833846 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:36.833797    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:36.836863 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:36.836846    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 00:55:36.848423 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:36.848390    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be4f8050-958b-471a-8bb9-765583b21e3d-original-pull-secret\") pod \"global-pull-secret-syncer-dsgtn\" (UID: \"be4f8050-958b-471a-8bb9-765583b21e3d\") " pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:36.900593 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:36.900570    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dsgtn"
Jul 29 00:55:36.934469 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:36.934442    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:55:36.937491 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:36.937466    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 00:55:36.944794 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:36.944762    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 00:55:36.944995 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:36.944974    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:56:40.944952911 +0000 UTC m=+130.275602509 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : secret "metrics-daemon-secret" not found
Jul 29 00:55:37.035051 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.035015    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:37.036500 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.036474    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dsgtn"]
Jul 29 00:55:37.037774 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.037749    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 00:55:37.039866 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:55:37.039846    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4f8050_958b_471a_8bb9_765583b21e3d.slice/crio-2e919532631560d868c77bcd0b552a8099d405d30682ab51004cc3faa0951688 WatchSource:0}: Error finding container 2e919532631560d868c77bcd0b552a8099d405d30682ab51004cc3faa0951688: Status 404 returned error can't find the container with id 2e919532631560d868c77bcd0b552a8099d405d30682ab51004cc3faa0951688
Jul 29 00:55:37.048151 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.048133    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 00:55:37.058319 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.058292    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8mc6\" (UniqueName: \"kubernetes.io/projected/bbea195e-3ebe-4216-8d89-8963e218f2d5-kube-api-access-h8mc6\") pod \"network-check-target-62pg4\" (UID: \"bbea195e-3ebe-4216-8d89-8963e218f2d5\") " pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:37.198982 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.198965    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ndg5h\""
Jul 29 00:55:37.207031 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.207014    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:37.356728 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.356687    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62pg4"]
Jul 29 00:55:37.359981 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:55:37.359952    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbea195e_3ebe_4216_8d89_8963e218f2d5.slice/crio-d1291bb158f19a68d14ad5ec1f774ba8638a1804dcde8f708a2d24728f284671 WatchSource:0}: Error finding container d1291bb158f19a68d14ad5ec1f774ba8638a1804dcde8f708a2d24728f284671: Status 404 returned error can't find the container with id d1291bb158f19a68d14ad5ec1f774ba8638a1804dcde8f708a2d24728f284671
Jul 29 00:55:37.423159 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.423129    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dsgtn" event={"ID":"be4f8050-958b-471a-8bb9-765583b21e3d","Type":"ContainerStarted","Data":"2e919532631560d868c77bcd0b552a8099d405d30682ab51004cc3faa0951688"}
Jul 29 00:55:37.424090 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.424062    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62pg4" event={"ID":"bbea195e-3ebe-4216-8d89-8963e218f2d5","Type":"ContainerStarted","Data":"d1291bb158f19a68d14ad5ec1f774ba8638a1804dcde8f708a2d24728f284671"}
Jul 29 00:55:37.941798 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:37.941762    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:55:37.942329 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:37.941932    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:55:37.942329 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:55:37.942012    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:56:09.941991475 +0000 UTC m=+99.272641062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:55:42.438148 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:42.438106    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62pg4" event={"ID":"bbea195e-3ebe-4216-8d89-8963e218f2d5","Type":"ContainerStarted","Data":"2d2cfd082de5f2b91fd71c893c8bc1cdc5afd1695af6a28810e9277822348147"}
Jul 29 00:55:42.438581 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:42.438197    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:55:42.439445 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:42.439422    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dsgtn" event={"ID":"be4f8050-958b-471a-8bb9-765583b21e3d","Type":"ContainerStarted","Data":"86e93390867a63871874edf53a1ff4e2081837cefc4bb9735ac62adb793a02d6"}
Jul 29 00:55:42.453460 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:55:42.453417    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-62pg4" podStartSLOduration=67.095261658 podStartE2EDuration="1m11.453403683s" totalImagesPullingTime="4.358142025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:42.452315862 +0000 UTC m=+71.782965458" watchObservedRunningTime="2026-07-29 00:55:42.453403683 +0000 UTC m=+71.784053288"
Jul 29 00:56:02.037706 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.037571    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dsgtn" podStartSLOduration=86.357035369 podStartE2EDuration="1m31.037553665s" totalImagesPullingTime="4.680518296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:55:42.465161463 +0000 UTC m=+71.795811070" watchObservedRunningTime="2026-07-29 00:56:02.037553665 +0000 UTC m=+91.368203271"
Jul 29 00:56:02.037706 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.037681    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"]
Jul 29 00:56:02.042319 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.042294    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.045900 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.045873    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 29 00:56:02.046609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.046578    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 00:56:02.046609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.046584    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 29 00:56:02.046786 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.046638    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 29 00:56:02.046786 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.046716    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 29 00:56:02.046891 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.046787    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 00:56:02.046891 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.046787    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 00:56:02.048648 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.048625    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"]
Jul 29 00:56:02.102229 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.102184    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-ca\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.102436 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.102256    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.102436 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.102341    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.102436 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.102382    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-hub\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.102436 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.102408    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zsj\" (UniqueName: \"kubernetes.io/projected/6da79b78-d286-447f-a96e-f655fe101ffb-kube-api-access-g7zsj\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.102583 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.102460    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da79b78-d286-447f-a96e-f655fe101ffb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.203745 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.203696    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-hub\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.203745 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.203750    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zsj\" (UniqueName: \"kubernetes.io/projected/6da79b78-d286-447f-a96e-f655fe101ffb-kube-api-access-g7zsj\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.203963 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.203772    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da79b78-d286-447f-a96e-f655fe101ffb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.203963 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.203892    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-ca\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.204061 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.203976    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.204061 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.204034    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.204528 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.204501    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da79b78-d286-447f-a96e-f655fe101ffb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.206373 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.206347    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.206373 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.206360    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-hub\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.206520 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.206392    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-ca\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.206520 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.206430    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6da79b78-d286-447f-a96e-f655fe101ffb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.210710 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.210684    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zsj\" (UniqueName: \"kubernetes.io/projected/6da79b78-d286-447f-a96e-f655fe101ffb-kube-api-access-g7zsj\") pod \"cluster-proxy-proxy-agent-5d5b56d544-f4hhn\" (UID: \"6da79b78-d286-447f-a96e-f655fe101ffb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.366861 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.366776    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 00:56:02.485527 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:02.485494    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"]
Jul 29 00:56:02.490741 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:56:02.490709    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6da79b78_d286_447f_a96e_f655fe101ffb.slice/crio-9edf7d34657bdeeb8af56b404859ab428b2ddb7751248d5b43895b2b66cc37d7 WatchSource:0}: Error finding container 9edf7d34657bdeeb8af56b404859ab428b2ddb7751248d5b43895b2b66cc37d7: Status 404 returned error can't find the container with id 9edf7d34657bdeeb8af56b404859ab428b2ddb7751248d5b43895b2b66cc37d7
Jul 29 00:56:03.490055 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:03.490022    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" event={"ID":"6da79b78-d286-447f-a96e-f655fe101ffb","Type":"ContainerStarted","Data":"9edf7d34657bdeeb8af56b404859ab428b2ddb7751248d5b43895b2b66cc37d7"}
Jul 29 00:56:09.953760 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:09.953729    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:56:09.954148 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:56:09.953886    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:56:09.954148 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:56:09.953961    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:57:13.953944389 +0000 UTC m=+163.284593976 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:56:12.521765 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:12.521726    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" event={"ID":"6da79b78-d286-447f-a96e-f655fe101ffb","Type":"ContainerStarted","Data":"6a1bc67872f059942eabdf48208ae038e44d5a7a0e2bddee02364c9353591f0e"}
Jul 29 00:56:12.521765 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:12.521768    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" event={"ID":"6da79b78-d286-447f-a96e-f655fe101ffb","Type":"ContainerStarted","Data":"b3d173d5eb7b2468e776b4721f78a64a41c37c86096a400a6ac58bcdb9646b9d"}
Jul 29 00:56:12.522221 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:12.521782    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" event={"ID":"6da79b78-d286-447f-a96e-f655fe101ffb","Type":"ContainerStarted","Data":"a256026b30681ca081c0216ab520064d1479e60a2853dbe80608ac77636dcba8"}
Jul 29 00:56:12.540127 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:12.540080    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" podStartSLOduration=1.554600953 podStartE2EDuration="10.540066688s" totalImagesPullingTime="8.985465735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:56:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:56:12.538396754 +0000 UTC m=+101.869046360" watchObservedRunningTime="2026-07-29 00:56:12.540066688 +0000 UTC m=+101.870716291"
Jul 29 00:56:13.443537 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:13.443506    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-62pg4"
Jul 29 00:56:40.957545 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:56:40.957501    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:56:40.958030 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:56:40.957624    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 00:56:40.958030 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:56:40.957694    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:42.957674402 +0000 UTC m=+252.288323985 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : secret "metrics-daemon-secret" not found
Jul 29 00:57:09.042534 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:57:09.042494    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hpq5j" podUID="1b15c048-7e7f-4ca3-8b69-8f23cab9d488"
Jul 29 00:57:09.193000 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:57:09.192963    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:57:09.676542 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:57:09.676513    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:57:13.973688 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:57:13.973634    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:57:13.974105 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:57:13.973775    2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 00:57:13.974105 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:57:13.973846    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls podName:1b15c048-7e7f-4ca3-8b69-8f23cab9d488 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:15.973830468 +0000 UTC m=+285.304480057 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls") pod "dns-default-hpq5j" (UID: "1b15c048-7e7f-4ca3-8b69-8f23cab9d488") : secret "dns-default-metrics-tls" not found
Jul 29 00:57:22.183229 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:57:22.183179    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:57:23.636149 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:57:23.636080    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:04.397692 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:04.397663    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:29.521211 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.521176    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"]
Jul 29 00:58:29.523861 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.523845    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"
Jul 29 00:58:29.526234 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.526215    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 00:58:29.527577 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.527563    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-g9rj7\""
Jul 29 00:58:29.527637 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.527606    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 00:58:29.536311 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.536291    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"]
Jul 29 00:58:29.581458 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.581431    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfr8q\" (UniqueName: \"kubernetes.io/projected/9f5f978a-057a-4967-9c93-d42238951f2c-kube-api-access-pfr8q\") pod \"volume-data-source-validator-549c6d54cf-gxfx4\" (UID: \"9f5f978a-057a-4967-9c93-d42238951f2c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"
Jul 29 00:58:29.682294 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.682245    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfr8q\" (UniqueName: \"kubernetes.io/projected/9f5f978a-057a-4967-9c93-d42238951f2c-kube-api-access-pfr8q\") pod \"volume-data-source-validator-549c6d54cf-gxfx4\" (UID: \"9f5f978a-057a-4967-9c93-d42238951f2c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"
Jul 29 00:58:29.689535 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.689514    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfr8q\" (UniqueName: \"kubernetes.io/projected/9f5f978a-057a-4967-9c93-d42238951f2c-kube-api-access-pfr8q\") pod \"volume-data-source-validator-549c6d54cf-gxfx4\" (UID: \"9f5f978a-057a-4967-9c93-d42238951f2c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"
Jul 29 00:58:29.832606 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.832525    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"
Jul 29 00:58:29.949961 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:29.949929    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4"]
Jul 29 00:58:29.953013 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:58:29.952982    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5f978a_057a_4967_9c93_d42238951f2c.slice/crio-7da7f8aeefaad6ae6c12095c744e80a4bbd4abd5ef8a376af7f0260f7a3c45e6 WatchSource:0}: Error finding container 7da7f8aeefaad6ae6c12095c744e80a4bbd4abd5ef8a376af7f0260f7a3c45e6: Status 404 returned error can't find the container with id 7da7f8aeefaad6ae6c12095c744e80a4bbd4abd5ef8a376af7f0260f7a3c45e6
Jul 29 00:58:30.147238 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:30.147146    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:30.247717 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:30.247684    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:30.895419 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:30.895382    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4" event={"ID":"9f5f978a-057a-4967-9c93-d42238951f2c","Type":"ContainerStarted","Data":"7da7f8aeefaad6ae6c12095c744e80a4bbd4abd5ef8a376af7f0260f7a3c45e6"}
Jul 29 00:58:31.899203 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:31.899164    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4" event={"ID":"9f5f978a-057a-4967-9c93-d42238951f2c","Type":"ContainerStarted","Data":"16f7f811d2659bb347a1318df70185ed82796ff40cadbd895aeb381aa7e4ff0a"}
Jul 29 00:58:31.913926 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:31.913882    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-gxfx4" podStartSLOduration=1.071924633 podStartE2EDuration="2.913867722s" totalImagesPullingTime="1.841943089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:58:31.912774475 +0000 UTC m=+241.243424080" watchObservedRunningTime="2026-07-29 00:58:31.913867722 +0000 UTC m=+241.244517310"
Jul 29 00:58:31.981265 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:31.981236    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:32.031649 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:32.031622    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:33.571455 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.571418    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hbpj8"]
Jul 29 00:58:33.574419 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.574403    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.577153 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.577129    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 00:58:33.578367 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.578350    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 00:58:33.578510 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.578363    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vj9dt\""
Jul 29 00:58:33.578510 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.578363    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 00:58:33.578510 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.578363    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 00:58:33.586289 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.586255    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hbpj8"]
Jul 29 00:58:33.595165 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.595148    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:33.652922 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.652896    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:33.713111 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.713085    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmnjd\" (UniqueName: \"kubernetes.io/projected/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-api-access-xmnjd\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.713198 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.713123    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.713198 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.713182    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/46fbc0f2-836f-43aa-8906-a91ec6c61b27-crio-socket\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.713269 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.713201    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.713269 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.713224    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/46fbc0f2-836f-43aa-8906-a91ec6c61b27-data-volume\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814361 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814335    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814460 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814363    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/46fbc0f2-836f-43aa-8906-a91ec6c61b27-data-volume\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814460 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814401    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmnjd\" (UniqueName: \"kubernetes.io/projected/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-api-access-xmnjd\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814460 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814440    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814460 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:33.814454    2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:33.814669 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814492    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/46fbc0f2-836f-43aa-8906-a91ec6c61b27-crio-socket\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814669 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:33.814528    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:34.3145093 +0000 UTC m=+243.645158898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:33.814669 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814561    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/46fbc0f2-836f-43aa-8906-a91ec6c61b27-crio-socket\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.814669 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:33.814658    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:33.814846 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:33.814716    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:34.31470001 +0000 UTC m=+243.645349594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:33.814846 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.814732    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/46fbc0f2-836f-43aa-8906-a91ec6c61b27-data-volume\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:33.824352 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:33.824296    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmnjd\" (UniqueName: \"kubernetes.io/projected/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-api-access-xmnjd\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:34.241055 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.241027    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:34.318427 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.318403    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:34.318511 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.318449    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:34.318560 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:34.318546    2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:34.318609 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:34.318599    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:35.318586536 +0000 UTC m=+244.649236120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:34.318651 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:34.318549    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:34.318686 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:34.318676    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:35.318664206 +0000 UTC m=+244.649313800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:34.340921 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.340900    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:34.630459 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.630391    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:34.729920 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.729891    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:34.830597 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.830567    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:34.943064 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:34.943037    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:35.042485 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:35.042456    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:35.325325 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:35.325238    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:35.325325 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:35.325312    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:35.325580 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:35.325346    2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:35.325580 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:35.325405    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:37.325387576 +0000 UTC m=+246.656037160 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:35.325580 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:35.325407    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:35.325580 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:35.325446    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:37.325435039 +0000 UTC m=+246.656084623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:35.745845 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:35.745821    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w44jh_cdc09b7a-c22b-4e09-b828-1b7a4ef8c289/dns-node-resolver/0.log"
Jul 29 00:58:36.578388 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:36.578354    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:36.745867 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:36.745839    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tsn5b_05209071-ba00-4003-8c93-5b47ca7c3d8f/node-ca/0.log"
Jul 29 00:58:37.337013 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:37.336980    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:37.337176 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:37.337030    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:37.337176 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:37.337103    2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:37.337176 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:37.337113    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:37.337176 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:37.337150    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:41.337137247 +0000 UTC m=+250.667786832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:37.337176 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:37.337162    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:41.337156308 +0000 UTC m=+250.667805892 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:37.345444 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:37.345426    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k2bfw_b249760f-1f36-4430-ab58-22f44d91f7a9/serve-healthcheck-canary/0.log"
Jul 29 00:58:37.961225 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:37.961194    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:39.816704 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:39.816615    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.018706 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.018673    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.157233 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.157146    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.261019 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.260989    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.378171 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.378140    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.687614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.687576    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.821945 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.821921    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.922351 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.922316    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:40.979401 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:40.979376    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:41.310127 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:41.310024    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:41.366727 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:41.366693    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:41.366914 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:41.366743    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:41.366914 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:41.366806    2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 00:58:41.366914 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:41.366849    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:41.366914 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:41.366858    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:49.366844481 +0000 UTC m=+258.697494064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : configmap "kube-rbac-proxy" not found
Jul 29 00:58:41.367080 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:41.366918    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:58:49.366899602 +0000 UTC m=+258.697549193 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:42.305496 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:42.305464    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:42.721150 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:42.721103    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:42.911377 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:42.911348    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:42.980492 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:42.980388    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:58:42.980657 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:42.980543    2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 00:58:42.980657 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:42.980611    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs podName:034fc553-7c4a-4f22-b467-df36e2e94071 nodeName:}" failed. No retries permitted until 2026-07-29 01:00:44.980592861 +0000 UTC m=+374.311242463 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs") pod "network-metrics-daemon-rb9df" (UID: "034fc553-7c4a-4f22-b467-df36e2e94071") : secret "metrics-daemon-secret" not found
Jul 29 00:58:43.112938 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:43.112905    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:43.435792 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:43.435718    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:43.535943 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:43.535915    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:43.730155 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:43.730131    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:43.964634 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:43.964603    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:44.310644 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:44.310614    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:44.708292 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:44.708247    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:44.914220 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:44.914165    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:45.111582 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:45.111491    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:45.307973 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:45.307931    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:46.293937 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:46.293873    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:46.339865 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:46.339831    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:46.708599 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:46.708572    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:46.808510 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:46.808485    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:46.909419 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:46.909397    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:47.109906 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:47.109821    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:47.509629 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:47.509596    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:48.308445 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:48.308408    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:48.711055 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:48.711028    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:48.914562 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:48.914521    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:49.114536 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:49.114442    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:49.428623 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:49.428530    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:49.428623 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:49.428600    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:49.428809 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:49.428704    2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:49.428809 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:58:49.428765    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls podName:46fbc0f2-836f-43aa-8906-a91ec6c61b27 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:05.42875094 +0000 UTC m=+274.759400524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hbpj8" (UID: "46fbc0f2-836f-43aa-8906-a91ec6c61b27") : secret "insights-runtime-extractor-tls" not found
Jul 29 00:58:49.429254 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:49.429235    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/46fbc0f2-836f-43aa-8906-a91ec6c61b27-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:58:49.511533 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:49.511507    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:50.110973 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:50.110947    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:50.336030 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:50.336001    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:50.513432 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:50.513389    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:55.118850 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:55.118754    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:55.171074 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:55.171033    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:58.349875 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.349846    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b4468786b-df8lr"]
Jul 29 00:58:58.352628 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.352612    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.355242 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.355222    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 00:58:58.355408 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.355388    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 00:58:58.355502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.355455    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6dlx2\""
Jul 29 00:58:58.355687 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.355673    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 00:58:58.361952 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.361929    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 00:58:58.369600 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.369581    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b4468786b-df8lr"]
Jul 29 00:58:58.407582 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.407559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:58.488268 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488241    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-installation-pull-secrets\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488396 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488330    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4b2q\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-kube-api-access-j4b2q\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488396 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488377    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-bound-sa-token\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488501 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488413    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-image-registry-private-configuration\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488501 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488432    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-registry-certificates\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488610 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488500    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-ca-trust-extracted\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488610 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488530    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-trusted-ca\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.488610 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.488564    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-registry-tls\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.506368 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.506345    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:58.589425 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-image-registry-private-configuration\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589425 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589428    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-registry-certificates\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589651 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589460    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-ca-trust-extracted\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589651 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589484    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-trusted-ca\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589651 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589519    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-registry-tls\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589651 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589550    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-installation-pull-secrets\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589651 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589590    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4b2q\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-kube-api-access-j4b2q\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589651 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589640    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-bound-sa-token\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.589960 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.589874    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-ca-trust-extracted\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.590448 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.590424    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-trusted-ca\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.590673 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.590651    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-registry-certificates\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.591914 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.591886    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-image-registry-private-configuration\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.591993 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.591944    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-installation-pull-secrets\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.592066 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.592048    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-registry-tls\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.602193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.602138    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-bound-sa-token\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.602675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.602658    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4b2q\" (UniqueName: \"kubernetes.io/projected/e1db99fd-6618-4071-9c40-c2c1e5dd47ef-kube-api-access-j4b2q\") pod \"image-registry-7b4468786b-df8lr\" (UID: \"e1db99fd-6618-4071-9c40-c2c1e5dd47ef\") " pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.605603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.605581    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:58:58.661995 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.661964    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.785433 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.785404    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b4468786b-df8lr"]
Jul 29 00:58:58.787993 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:58:58.787968    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1db99fd_6618_4071_9c40_c2c1e5dd47ef.slice/crio-63fa0aee2cec38f5d832e46ebc198c93ead750fb7c4779d332e21e7904ef8f41 WatchSource:0}: Error finding container 63fa0aee2cec38f5d832e46ebc198c93ead750fb7c4779d332e21e7904ef8f41: Status 404 returned error can't find the container with id 63fa0aee2cec38f5d832e46ebc198c93ead750fb7c4779d332e21e7904ef8f41
Jul 29 00:58:58.970776 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.970740    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b4468786b-df8lr" event={"ID":"e1db99fd-6618-4071-9c40-c2c1e5dd47ef","Type":"ContainerStarted","Data":"a7d50fad328cf6550aa8b4abf295fe8a432dcb2d64392ab6b6f842e34ba3421c"}
Jul 29 00:58:58.970935 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.970792    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b4468786b-df8lr" event={"ID":"e1db99fd-6618-4071-9c40-c2c1e5dd47ef","Type":"ContainerStarted","Data":"63fa0aee2cec38f5d832e46ebc198c93ead750fb7c4779d332e21e7904ef8f41"}
Jul 29 00:58:58.970935 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.970822    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:58:58.989405 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:58:58.989360    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b4468786b-df8lr" podStartSLOduration=0.989346835 podStartE2EDuration="989.346835ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:58:58.988209486 +0000 UTC m=+268.318859118" watchObservedRunningTime="2026-07-29 00:58:58.989346835 +0000 UTC m=+268.319996469"
Jul 29 00:59:03.346683 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:03.346650    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:03.456107 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:03.456070    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:03.528131 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:03.528100    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:03.754011 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:03.753981    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:05.437887 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:05.437853    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:59:05.440094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:05.440073    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/46fbc0f2-836f-43aa-8906-a91ec6c61b27-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hbpj8\" (UID: \"46fbc0f2-836f-43aa-8906-a91ec6c61b27\") " pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:59:05.683600 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:05.683551    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hbpj8"
Jul 29 00:59:05.806913 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:05.806885    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hbpj8"]
Jul 29 00:59:05.810327 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:05.810304    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fbc0f2_836f_43aa_8906_a91ec6c61b27.slice/crio-f719199a6fb5b4fff78250303713ef7353bfee9ebdcaba856256c029d199e11a WatchSource:0}: Error finding container f719199a6fb5b4fff78250303713ef7353bfee9ebdcaba856256c029d199e11a: Status 404 returned error can't find the container with id f719199a6fb5b4fff78250303713ef7353bfee9ebdcaba856256c029d199e11a
Jul 29 00:59:05.991399 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:05.991320    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbpj8" event={"ID":"46fbc0f2-836f-43aa-8906-a91ec6c61b27","Type":"ContainerStarted","Data":"f719199a6fb5b4fff78250303713ef7353bfee9ebdcaba856256c029d199e11a"}
Jul 29 00:59:06.554037 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:06.553969    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:06.653770 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:06.653735    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:07.146947 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:07.146898    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:11.009946 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:11.009865    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbpj8" event={"ID":"46fbc0f2-836f-43aa-8906-a91ec6c61b27","Type":"ContainerStarted","Data":"9d83387001850a6ca5868b4704d11e4455ed78ba9a0fea2a1f5f28ef3a8f0913"}
Jul 29 00:59:11.110889 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:11.110810    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:12.677537 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:59:12.677500    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hpq5j" podUID="1b15c048-7e7f-4ca3-8b69-8f23cab9d488"
Jul 29 00:59:12.691603 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.691568    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:12.953086 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.953044    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"]
Jul 29 00:59:12.974183 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.974150    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9pgl2"]
Jul 29 00:59:12.974334 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.974288    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:12.977570 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.977542    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 00:59:12.978556 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.978532    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 00:59:12.978675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.978561    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 00:59:12.978675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.978589    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 00:59:12.978675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.978561    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2kd6r\""
Jul 29 00:59:12.978675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.978646    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 00:59:12.978854 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.978538    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 00:59:12.979116 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.979027    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:12.981630 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.981594    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 00:59:12.981735 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.981598    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f2tnk\""
Jul 29 00:59:12.981897 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.981870    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 00:59:12.981989 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.981968    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 00:59:12.994308 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:12.994288    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"]
Jul 29 00:59:13.015872 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.015853    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:59:13.097317 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097283    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097486 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097333    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097486 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097365    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-textfile\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097486 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097393    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-sys\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097627 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097507    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29rmh\" (UniqueName: \"kubernetes.io/projected/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-kube-api-access-29rmh\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097627 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097538    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.097627 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097560    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4c5caf5-64c2-4de7-a417-bf0497acbe98-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.097627 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097581    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.097818 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097637    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-root\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097818 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097666    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.097818 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4c5caf5-64c2-4de7-a417-bf0497acbe98-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.097818 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097772    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-wtmp\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.097818 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097804    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-metrics-client-ca\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.098022 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097837    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt85b\" (UniqueName: \"kubernetes.io/projected/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-api-access-gt85b\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.098022 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.097870    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-tls\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.198675 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.198647    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29rmh\" (UniqueName: \"kubernetes.io/projected/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-kube-api-access-29rmh\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.198850 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.198681    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.198850 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.198709    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4c5caf5-64c2-4de7-a417-bf0497acbe98-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.198850 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.198758    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.198850 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:59:13.198825    2555 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 00:59:13.199051 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:59:13.198899    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-tls podName:e4c5caf5-64c2-4de7-a417-bf0497acbe98 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:13.698877634 +0000 UTC m=+283.029527233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-b8mvr" (UID: "e4c5caf5-64c2-4de7-a417-bf0497acbe98") : secret "kube-state-metrics-tls" not found
Jul 29 00:59:13.199051 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.198948    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-root\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199051 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.198991    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.199051 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199022    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-root\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199051 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199034    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4c5caf5-64c2-4de7-a417-bf0497acbe98-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.199262 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199097    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-wtmp\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199262 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199133    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-metrics-client-ca\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199262 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199165    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt85b\" (UniqueName: \"kubernetes.io/projected/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-api-access-gt85b\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.199262 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199198    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-tls\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199262 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199258    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199259    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-wtmp\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199308    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199340    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-textfile\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199368    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-sys\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199414    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4c5caf5-64c2-4de7-a417-bf0497acbe98-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199421    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199460    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e4c5caf5-64c2-4de7-a417-bf0497acbe98-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.199614 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199473    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-sys\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.200036 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.199749    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-textfile\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.200177 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.200154    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-metrics-client-ca\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.200252 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.200229    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-accelerators-collector-config\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.201969 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.201949    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-tls\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.202536 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.202516    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.202617 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.202602    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.209672 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.209619    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29rmh\" (UniqueName: \"kubernetes.io/projected/b6c0d77a-5903-4505-b25d-b0d1fcdae4c2-kube-api-access-29rmh\") pod \"node-exporter-9pgl2\" (UID: \"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2\") " pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.209828 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.209812    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt85b\" (UniqueName: \"kubernetes.io/projected/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-api-access-gt85b\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.293190 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.293159    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9pgl2"
Jul 29 00:59:13.703006 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.702956    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.705665 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.705638    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e4c5caf5-64c2-4de7-a417-bf0497acbe98-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-b8mvr\" (UID: \"e4c5caf5-64c2-4de7-a417-bf0497acbe98\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:13.856770 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:13.856734    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6c0d77a_5903_4505_b25d_b0d1fcdae4c2.slice/crio-920f0166d59547df8e2180a5e748b00344fdd8b530ccb42c407ac55c2e3539d5 WatchSource:0}: Error finding container 920f0166d59547df8e2180a5e748b00344fdd8b530ccb42c407ac55c2e3539d5: Status 404 returned error can't find the container with id 920f0166d59547df8e2180a5e748b00344fdd8b530ccb42c407ac55c2e3539d5
Jul 29 00:59:13.887561 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:13.887535    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"
Jul 29 00:59:14.018480 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:14.018453    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr"]
Jul 29 00:59:14.020258 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:14.020232    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pgl2" event={"ID":"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2","Type":"ContainerStarted","Data":"920f0166d59547df8e2180a5e748b00344fdd8b530ccb42c407ac55c2e3539d5"}
Jul 29 00:59:14.021610 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:14.021583    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c5caf5_64c2_4de7_a417_bf0497acbe98.slice/crio-32f5751df8b9951df888dba15afbd46dc87eb33163b20b7b580eeae6f48d61a9 WatchSource:0}: Error finding container 32f5751df8b9951df888dba15afbd46dc87eb33163b20b7b580eeae6f48d61a9: Status 404 returned error can't find the container with id 32f5751df8b9951df888dba15afbd46dc87eb33163b20b7b580eeae6f48d61a9
Jul 29 00:59:14.021868 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:14.021850    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbpj8" event={"ID":"46fbc0f2-836f-43aa-8906-a91ec6c61b27","Type":"ContainerStarted","Data":"93f78482045542e8a442f672c46ffda76fbd124ca9bbc2b54a239db3acd71c96"}
Jul 29 00:59:14.697898 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:14.697861    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:15.027564 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:15.027481    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr" event={"ID":"e4c5caf5-64c2-4de7-a417-bf0497acbe98","Type":"ContainerStarted","Data":"32f5751df8b9951df888dba15afbd46dc87eb33163b20b7b580eeae6f48d61a9"}
Jul 29 00:59:15.871226 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:15.871150    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:15.970873 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:15.970804    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:16.023719 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.023680    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:59:16.026229 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.026202    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b15c048-7e7f-4ca3-8b69-8f23cab9d488-metrics-tls\") pod \"dns-default-hpq5j\" (UID: \"1b15c048-7e7f-4ca3-8b69-8f23cab9d488\") " pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:59:16.034077 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.034017    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr" event={"ID":"e4c5caf5-64c2-4de7-a417-bf0497acbe98","Type":"ContainerStarted","Data":"3572f8a1e09862d58cf753ab2a23093e3e0782a169ec2d8cb59f09979dcf1534"}
Jul 29 00:59:16.034487 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.034127    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr" event={"ID":"e4c5caf5-64c2-4de7-a417-bf0497acbe98","Type":"ContainerStarted","Data":"eea8fa7ce02fdb2c41552889e9b79368663b78cc4811ef8ffc98decdac890d56"}
Jul 29 00:59:16.034487 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.034165    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr" event={"ID":"e4c5caf5-64c2-4de7-a417-bf0497acbe98","Type":"ContainerStarted","Data":"9bebb255a69bf615695e358cd8b71c8305be55f4d290d01d1740769897377f7a"}
Jul 29 00:59:16.036404 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.036346    2555 generic.go:356] "Generic (PLEG): container finished" podID="b6c0d77a-5903-4505-b25d-b0d1fcdae4c2" containerID="8aa785c2d0d7665df721ea26e9864cce531e4a9a01bb362c862e0b4774859d77" exitCode=0
Jul 29 00:59:16.036404 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.036376    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pgl2" event={"ID":"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2","Type":"ContainerDied","Data":"8aa785c2d0d7665df721ea26e9864cce531e4a9a01bb362c862e0b4774859d77"}
Jul 29 00:59:16.052562 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.052516    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-b8mvr" podStartSLOduration=2.439227296 podStartE2EDuration="4.052498906s" totalImagesPullingTime="1.61327161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:16.052072804 +0000 UTC m=+285.382722412" watchObservedRunningTime="2026-07-29 00:59:16.052498906 +0000 UTC m=+285.383148511"
Jul 29 00:59:16.253802 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.253764    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:16.319243 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.319216    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9mcp8\""
Jul 29 00:59:16.327532 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.327511    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:59:16.461707 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.461555    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hpq5j"]
Jul 29 00:59:16.487546 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:16.487514    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:17.042965 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.042927    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pgl2" event={"ID":"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2","Type":"ContainerStarted","Data":"6a6c0f4c6701f1988fe53f6f8020e5b534861007e1caf6f8dd6a4e8f0a35a1d6"}
Jul 29 00:59:17.043445 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.042992    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9pgl2" event={"ID":"b6c0d77a-5903-4505-b25d-b0d1fcdae4c2","Type":"ContainerStarted","Data":"c0cb004fe36816a0f993976edf7284db19a1b160cd23dd151d09989ba4efd921"}
Jul 29 00:59:17.063104 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.063058    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9pgl2" podStartSLOduration=3.196808001 podStartE2EDuration="5.063042667s" totalImagesPullingTime="1.778741517s" totalInitContainerRuntime="87.493149ms" isStatefulPod=true podCreationTimestamp="2026-07-29 00:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:17.062180213 +0000 UTC m=+286.392829822" watchObservedRunningTime="2026-07-29 00:59:17.063042667 +0000 UTC m=+286.393692272"
Jul 29 00:59:17.245800 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.245766    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"]
Jul 29 00:59:17.260066 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.260037    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"]
Jul 29 00:59:17.260247 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.260161    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.262786 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.262758    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 00:59:17.263345 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.263004    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fmtsv9tfu8mtp\""
Jul 29 00:59:17.263345 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.263163    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 00:59:17.263345 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.263208    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q5q9g\""
Jul 29 00:59:17.263572 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.263411    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 00:59:17.263652 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.263634    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 00:59:17.312193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.312105    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:17.334351 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334323    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-client-ca-bundle\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.334502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334361    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-secret-metrics-server-tls\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.334502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334383    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aef8e36c-d565-4672-9210-99ec6d3cfffe-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.334502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334415    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-secret-metrics-server-client-certs\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.334502 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334488    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/aef8e36c-d565-4672-9210-99ec6d3cfffe-audit-log\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.334679 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334523    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj249\" (UniqueName: \"kubernetes.io/projected/aef8e36c-d565-4672-9210-99ec6d3cfffe-kube-api-access-dj249\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.334679 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.334545    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/aef8e36c-d565-4672-9210-99ec6d3cfffe-metrics-server-audit-profiles\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.369107 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.369073    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:17.435247 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435216    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-client-ca-bundle\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435439 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435259    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-secret-metrics-server-tls\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435439 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435302    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aef8e36c-d565-4672-9210-99ec6d3cfffe-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435439 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435331    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-secret-metrics-server-client-certs\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435439 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435368    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/aef8e36c-d565-4672-9210-99ec6d3cfffe-audit-log\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435655 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435508    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj249\" (UniqueName: \"kubernetes.io/projected/aef8e36c-d565-4672-9210-99ec6d3cfffe-kube-api-access-dj249\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435655 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435564    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/aef8e36c-d565-4672-9210-99ec6d3cfffe-metrics-server-audit-profiles\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.435774 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.435738    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/aef8e36c-d565-4672-9210-99ec6d3cfffe-audit-log\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.436182 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.436155    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aef8e36c-d565-4672-9210-99ec6d3cfffe-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.436660 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.436631    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/aef8e36c-d565-4672-9210-99ec6d3cfffe-metrics-server-audit-profiles\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.438020 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.437979    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-secret-metrics-server-tls\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.438203 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.438157    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-client-ca-bundle\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.438343 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.438219    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/aef8e36c-d565-4672-9210-99ec6d3cfffe-secret-metrics-server-client-certs\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.443858 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.443834    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj249\" (UniqueName: \"kubernetes.io/projected/aef8e36c-d565-4672-9210-99ec6d3cfffe-kube-api-access-dj249\") pod \"metrics-server-7d9b786d7f-6vtbt\" (UID: \"aef8e36c-d565-4672-9210-99ec6d3cfffe\") " pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.469976 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.469946    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:17.571256 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.571188    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:17.573100 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.573079    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:17.713484 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.713448    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"]
Jul 29 00:59:17.716860 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.716828    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:17.720187 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.720167    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:17.722973 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.722950    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-xvth5\""
Jul 29 00:59:17.723061 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.722950    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 00:59:17.724208 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.724182    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"]
Jul 29 00:59:17.737839 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.737762    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsdh7\" (UniqueName: \"kubernetes.io/projected/a117de77-4eed-4c31-83f0-d533610d2a71-kube-api-access-zsdh7\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:17.737971 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.737909    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a117de77-4eed-4c31-83f0-d533610d2a71-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:17.839373 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.839287    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsdh7\" (UniqueName: \"kubernetes.io/projected/a117de77-4eed-4c31-83f0-d533610d2a71-kube-api-access-zsdh7\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:17.839536 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.839401    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a117de77-4eed-4c31-83f0-d533610d2a71-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:17.839585 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:59:17.839554    2555 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 29 00:59:17.839638 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:59:17.839625    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a117de77-4eed-4c31-83f0-d533610d2a71-monitoring-plugin-cert podName:a117de77-4eed-4c31-83f0-d533610d2a71 nodeName:}" failed. No retries permitted until 2026-07-29 00:59:18.339604035 +0000 UTC m=+287.670253624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a117de77-4eed-4c31-83f0-d533610d2a71-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-5t97k" (UID: "a117de77-4eed-4c31-83f0-d533610d2a71") : secret "monitoring-plugin-cert" not found
Jul 29 00:59:17.859785 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:17.859757    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsdh7\" (UniqueName: \"kubernetes.io/projected/a117de77-4eed-4c31-83f0-d533610d2a71-kube-api-access-zsdh7\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:18.050794 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:18.050766    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b15c048_7e7f_4ca3_8b69_8f23cab9d488.slice/crio-334cfea3c4ffd192ebbc224b3185dab2819271e93247fa09df418b986e136406 WatchSource:0}: Error finding container 334cfea3c4ffd192ebbc224b3185dab2819271e93247fa09df418b986e136406: Status 404 returned error can't find the container with id 334cfea3c4ffd192ebbc224b3185dab2819271e93247fa09df418b986e136406
Jul 29 00:59:18.180629 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.180546    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"]
Jul 29 00:59:18.182609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.182574    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:18.183597 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:18.183577    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef8e36c_d565_4672_9210_99ec6d3cfffe.slice/crio-73966a28a858d07f8c16f684ae3b832f8965c72ef6b47f516903993308fa4770 WatchSource:0}: Error finding container 73966a28a858d07f8c16f684ae3b832f8965c72ef6b47f516903993308fa4770: Status 404 returned error can't find the container with id 73966a28a858d07f8c16f684ae3b832f8965c72ef6b47f516903993308fa4770
Jul 29 00:59:18.344788 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.344722    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a117de77-4eed-4c31-83f0-d533610d2a71-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:18.346755 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.346735    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a117de77-4eed-4c31-83f0-d533610d2a71-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-5t97k\" (UID: \"a117de77-4eed-4c31-83f0-d533610d2a71\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:18.428116 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.428093    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:18.477310 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.477267    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:18.631582 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.631507    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:18.765356 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:18.765323    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"]
Jul 29 00:59:18.769601 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:18.769575    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda117de77_4eed_4c31_83f0_d533610d2a71.slice/crio-60721e7b8047a70da297edb974a600038bda3d4d50dc40149e2b0f139950ac67 WatchSource:0}: Error finding container 60721e7b8047a70da297edb974a600038bda3d4d50dc40149e2b0f139950ac67: Status 404 returned error can't find the container with id 60721e7b8047a70da297edb974a600038bda3d4d50dc40149e2b0f139950ac67
Jul 29 00:59:19.054744 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.054705    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k" event={"ID":"a117de77-4eed-4c31-83f0-d533610d2a71","Type":"ContainerStarted","Data":"60721e7b8047a70da297edb974a600038bda3d4d50dc40149e2b0f139950ac67"}
Jul 29 00:59:19.055911 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.055883    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt" event={"ID":"aef8e36c-d565-4672-9210-99ec6d3cfffe","Type":"ContainerStarted","Data":"73966a28a858d07f8c16f684ae3b832f8965c72ef6b47f516903993308fa4770"}
Jul 29 00:59:19.057117 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.057092    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hpq5j" event={"ID":"1b15c048-7e7f-4ca3-8b69-8f23cab9d488","Type":"ContainerStarted","Data":"334cfea3c4ffd192ebbc224b3185dab2819271e93247fa09df418b986e136406"}
Jul 29 00:59:19.059251 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.059226    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hbpj8" event={"ID":"46fbc0f2-836f-43aa-8906-a91ec6c61b27","Type":"ContainerStarted","Data":"b094893e35665df10c040b181eec88e27dc47e2211c66a84967f8d33c76c7fe9"}
Jul 29 00:59:19.077840 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.077799    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hbpj8" podStartSLOduration=33.913905169 podStartE2EDuration="46.077784735s" totalImagesPullingTime="12.163879566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 00:58:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:19.075875695 +0000 UTC m=+288.406525302" watchObservedRunningTime="2026-07-29 00:59:19.077784735 +0000 UTC m=+288.408434340"
Jul 29 00:59:19.148813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.148776    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 00:59:19.153336 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.152939    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.156793 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.156745    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 00:59:19.156793 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.156774    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 00:59:19.156990 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.156774    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 00:59:19.156990 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.156784    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2u6ovpdqgcrk1\""
Jul 29 00:59:19.157767 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.157746    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 00:59:19.158265 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.158241    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 00:59:19.158585 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.158467    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 00:59:19.158685 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.158594    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 00:59:19.158685 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.158679    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 00:59:19.159390 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.158978    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 00:59:19.159390 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.159028    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 00:59:19.159390 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.159061    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 00:59:19.159390 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.159308    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-56ssq\""
Jul 29 00:59:19.159659 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.159489    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 00:59:19.160040 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.160006    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 00:59:19.160806 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.160616    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 00:59:19.171600 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.171578    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 00:59:19.175637 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.175551    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:19.253901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.253828    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.253901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.253866    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.253901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.253898    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.253961    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254026    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254053    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254077    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254119    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254144    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254193 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254171    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254201    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254256    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254289    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5svr\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-kube-api-access-d5svr\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254324    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254349    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254437    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254481    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254506    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254534    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.254609 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.254557    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355226 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355150    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355226 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355189    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355226 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355206    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355236    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355263    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355329    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355355    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355401    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355441    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355480    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355509    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355538    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355573    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355596    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5svr\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-kube-api-access-d5svr\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355629    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355651    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.355813 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355729    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.356133 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355878    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.356133 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355913    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.356133 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.355947    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.356810 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.356519    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.356810 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.356543    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.357015 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.356906    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.357095 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.357071    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.357150 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.357134    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.357977 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.357946    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.358514 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.358489    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.360026 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.359965    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.360149 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.360128    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.360303 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.360224    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.360611 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.360585    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.360809 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.360785    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.361009 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.360980    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.361414 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.361391    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.362143 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.362104    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.362780 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.362759    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.363489 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.363454    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.363663 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.363638    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.363863 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.363846    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.367875 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.367846    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5svr\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-kube-api-access-d5svr\") pod \"prometheus-k8s-0\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.465995 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.465960    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:19.979454 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:19.979426    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b4468786b-df8lr"
Jul 29 00:59:20.105685 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:20.105648    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:20.205255 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:20.205153    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:20.279323 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:20.279292    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 00:59:20.290019 ip-10-0-139-6 kubenswrapper[2555]: W0729 00:59:20.289990    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd852339b_8569_453f_bdfc_bcf2e3bb69e8.slice/crio-cebbb145d233a25ff17cd636287211bb4c6584d17419d8a14429a4bb95ce2e69 WatchSource:0}: Error finding container cebbb145d233a25ff17cd636287211bb4c6584d17419d8a14429a4bb95ce2e69: Status 404 returned error can't find the container with id cebbb145d233a25ff17cd636287211bb4c6584d17419d8a14429a4bb95ce2e69
Jul 29 00:59:21.068093 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.068055    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k" event={"ID":"a117de77-4eed-4c31-83f0-d533610d2a71","Type":"ContainerStarted","Data":"121a23c5184bd849b28db5e003bf1d10653775c2c8a14c1cab7dd46373e188ee"}
Jul 29 00:59:21.068266 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.068247    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:21.069693 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.069665    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt" event={"ID":"aef8e36c-d565-4672-9210-99ec6d3cfffe","Type":"ContainerStarted","Data":"35a0552838b7392ef6799532217991a3fa8a466770c2d4feddda0286f20d23fd"}
Jul 29 00:59:21.071428 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.071390    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hpq5j" event={"ID":"1b15c048-7e7f-4ca3-8b69-8f23cab9d488","Type":"ContainerStarted","Data":"7af774a7acd5727979fb87881c78c713b3f73f339f686690dc9af354daea8a3f"}
Jul 29 00:59:21.071535 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.071442    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hpq5j" event={"ID":"1b15c048-7e7f-4ca3-8b69-8f23cab9d488","Type":"ContainerStarted","Data":"45488e3ac4a43ae7cc1a75a4e4d7cf3327f1fb6124ef34e8fc2078c10ce8b7c5"}
Jul 29 00:59:21.071535 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.071472    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:59:21.072548 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.072527    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"cebbb145d233a25ff17cd636287211bb4c6584d17419d8a14429a4bb95ce2e69"}
Jul 29 00:59:21.073674 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.073656    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k"
Jul 29 00:59:21.084630 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.084587    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-5t97k" podStartSLOduration=2.1872979790000002 podStartE2EDuration="4.084571296s" totalImagesPullingTime="1.897273317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:21.08410613 +0000 UTC m=+290.414755735" watchObservedRunningTime="2026-07-29 00:59:21.084571296 +0000 UTC m=+290.415220881"
Jul 29 00:59:21.119178 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.119129    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hpq5j" podStartSLOduration=253.05650909 podStartE2EDuration="4m15.11911605s" totalImagesPullingTime="2.06260696s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:55:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:21.117740271 +0000 UTC m=+290.448389878" watchObservedRunningTime="2026-07-29 00:59:21.11911605 +0000 UTC m=+290.449765656"
Jul 29 00:59:21.135063 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:21.135009    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt" podStartSLOduration=2.201890698 podStartE2EDuration="4.134989812s" totalImagesPullingTime="1.933099114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:21.133649606 +0000 UTC m=+290.464299213" watchObservedRunningTime="2026-07-29 00:59:21.134989812 +0000 UTC m=+290.465639419"
Jul 29 00:59:23.082172 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:23.082136    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790" exitCode=0
Jul 29 00:59:23.082659 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:23.082227    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790"}
Jul 29 00:59:25.120983 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:25.120942    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:25.183976 ip-10-0-139-6 kubenswrapper[2555]: E0729 00:59:25.183942    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rb9df" podUID="034fc553-7c4a-4f22-b467-df36e2e94071"
Jul 29 00:59:25.594146 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:25.594105    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:25.654901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:25.654861    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:25.784682 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:25.784646    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:25.885531 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:25.885430    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:25.988989 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:25.988950    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:26.194085 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:26.194052    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:26.793994 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:26.793898    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:26.998814 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:26.998788    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:27.097901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:27.097824    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53"}
Jul 29 00:59:27.097901 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:27.097885    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465"}
Jul 29 00:59:27.900016 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:27.899982    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:28.000849 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:28.000815    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:29.108867 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:29.108831    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9"}
Jul 29 00:59:29.109247 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:29.108893    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d"}
Jul 29 00:59:29.109247 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:29.108920    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084"}
Jul 29 00:59:29.109247 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:29.108943    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerStarted","Data":"979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c"}
Jul 29 00:59:29.139479 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:29.139424    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.140275994 podStartE2EDuration="10.139405936s" totalImagesPullingTime="6.981828106s" totalInitContainerRuntime="17.301836ms" isStatefulPod=false podCreationTimestamp="2026-07-29 00:59:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 00:59:29.137557392 +0000 UTC m=+298.468206998" watchObservedRunningTime="2026-07-29 00:59:29.139405936 +0000 UTC m=+298.470055543"
Jul 29 00:59:29.466261 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:29.466223    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 00:59:30.650741 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:30.650708    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:30.713927 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:30.713897    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:31.063384 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:31.063355    2555 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 00:59:31.079090 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:31.079065    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hpq5j"
Jul 29 00:59:33.125451 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:33.125413    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:33.174927 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:33.174899    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:34.610152 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:34.610117    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:34.711137 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:34.711111    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:34.819656 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:34.819625    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:34.920895 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:34.920826    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:37.183702 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:37.183668    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 00:59:37.573915 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:37.573881    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:37.574135 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:37.573937    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:39.757032 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:39.756996    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:39.858203 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:39.858131    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:40.021171 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:40.021140    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:40.121950 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:40.121879    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:40.222429 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:40.222399    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:42.368866 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:42.368820    2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" podUID="6da79b78-d286-447f-a96e-f655fe101ffb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 29 00:59:43.418110 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:43.418077    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:43.481743 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:43.481711    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:43.608921 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:43.608887    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:43.708748 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:43.708718    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:47.629771 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:47.629735    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:47.684859 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:47.684822    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:48.622036 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:48.622005    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:48.671900 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:48.671869    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:52.368735 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:52.368695    2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" podUID="6da79b78-d286-447f-a96e-f655fe101ffb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 29 00:59:52.624094 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:52.624014    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:52.680582 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:52.680553    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:54.777702 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:54.777668    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:54.838449 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:54.838411    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:55.122107 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:55.122031    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:55.185019 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:55.184987    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 00:59:57.578557 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:57.578528    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 00:59:57.582261 ip-10-0-139-6 kubenswrapper[2555]: I0729 00:59:57.582241    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7d9b786d7f-6vtbt"
Jul 29 01:00:02.368112 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:02.368071    2555 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" podUID="6da79b78-d286-447f-a96e-f655fe101ffb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 29 01:00:02.368498 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:02.368142    2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn"
Jul 29 01:00:02.368724 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:02.368691    2555 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"6a1bc67872f059942eabdf48208ae038e44d5a7a0e2bddee02364c9353591f0e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 29 01:00:02.368761 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:02.368747    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" podUID="6da79b78-d286-447f-a96e-f655fe101ffb" containerName="service-proxy" containerID="cri-o://6a1bc67872f059942eabdf48208ae038e44d5a7a0e2bddee02364c9353591f0e" gracePeriod=30
Jul 29 01:00:03.235935 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:03.235894    2555 generic.go:356] "Generic (PLEG): container finished" podID="6da79b78-d286-447f-a96e-f655fe101ffb" containerID="6a1bc67872f059942eabdf48208ae038e44d5a7a0e2bddee02364c9353591f0e" exitCode=2
Jul 29 01:00:03.236110 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:03.235945    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" event={"ID":"6da79b78-d286-447f-a96e-f655fe101ffb","Type":"ContainerDied","Data":"6a1bc67872f059942eabdf48208ae038e44d5a7a0e2bddee02364c9353591f0e"}
Jul 29 01:00:03.236110 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:03.235982    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d5b56d544-f4hhn" event={"ID":"6da79b78-d286-447f-a96e-f655fe101ffb","Type":"ContainerStarted","Data":"cc005124c09a5d003cfb0507c29e1b9182a2120f57cd58d7663b1c65be1609d0"}
Jul 29 01:00:19.467149 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:19.467110    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:19.485964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:19.485939    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:20.309077 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:20.309050    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:25.762154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:25.762076    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:25.863309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:25.863252    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:29.034441 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:29.034408    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:29.135691 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:29.135664    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:37.558297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.554855    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 01:00:37.559235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.558906    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="prometheus" containerID="cri-o://7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465" gracePeriod=600
Jul 29 01:00:37.559235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.558948    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy" containerID="cri-o://9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d" gracePeriod=600
Jul 29 01:00:37.559235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.558952    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-web" containerID="cri-o://4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084" gracePeriod=600
Jul 29 01:00:37.559235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.559129    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="config-reloader" containerID="cri-o://48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53" gracePeriod=600
Jul 29 01:00:37.559235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.559159    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="thanos-sidecar" containerID="cri-o://979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c" gracePeriod=600
Jul 29 01:00:37.559235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.559194    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9" gracePeriod=600
Jul 29 01:00:37.560406 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.560378    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:37.814130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:37.814054    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:38.358424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358388    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9" exitCode=0
Jul 29 01:00:38.358424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358410    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d" exitCode=0
Jul 29 01:00:38.358424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358416    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c" exitCode=0
Jul 29 01:00:38.358424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358421    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53" exitCode=0
Jul 29 01:00:38.358424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358427    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465" exitCode=0
Jul 29 01:00:38.358700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358456    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9"}
Jul 29 01:00:38.358700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358514    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d"}
Jul 29 01:00:38.358700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358540    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c"}
Jul 29 01:00:38.358700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358565    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53"}
Jul 29 01:00:38.358700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.358590    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465"}
Jul 29 01:00:38.807545 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.807523    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:38.913414 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913339    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913414 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913376    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-tls\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913414 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913397    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-serving-certs-ca-bundle\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913416    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-0\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913444    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-1\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913472    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-grpc-tls\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913496    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5svr\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-kube-api-access-d5svr\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913534    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-metrics-client-certs\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913558    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-kubelet-serving-ca-bundle\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913586    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-2\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.913654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913639    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-kube-rbac-proxy\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913675    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-trusted-ca-bundle\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913708    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-web-config\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913746    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913785    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-thanos-prometheus-http-client-file\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913818    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-tls-assets\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913844    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913868    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-db\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913878    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913893    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config-out\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913924    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-metrics-client-ca\") pod \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\" (UID: \"d852339b-8569-453f-bdfc-bcf2e3bb69e8\") "
Jul 29 01:00:38.914044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.913961    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.914625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.914081    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.914625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.914215    2555 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:38.914625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.914236    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:38.914625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.914343    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:38.914625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.914456    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.915401 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.914940    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.915941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.915664    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.916258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.916223    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:00:38.916370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.916324    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.917108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.916828    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:00:38.917108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.916922    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-kube-api-access-d5svr" (OuterVolumeSpecName: "kube-api-access-d5svr") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "kube-api-access-d5svr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:00:38.917108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.916982    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.917108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.917071    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.917495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.917456    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.917713 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.917687    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.918132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.918105    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:00:38.918732 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.918701    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.918810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.918727    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.919077 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.919052    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config-out" (OuterVolumeSpecName: "config-out") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:00:38.919386 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.919362    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config" (OuterVolumeSpecName: "config") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:38.928251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:38.928230    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-web-config" (OuterVolumeSpecName: "web-config") pod "d852339b-8569-453f-bdfc-bcf2e3bb69e8" (UID: "d852339b-8569-453f-bdfc-bcf2e3bb69e8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:00:39.015142 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015093    2555 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-metrics-client-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015142 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015137    2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015142 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015149    2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015142 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015157    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015166    2555 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-grpc-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015174    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5svr\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-kube-api-access-d5svr\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015182    2555 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-metrics-client-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015190    2555 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015199    2555 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-kube-rbac-proxy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015207    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015215    2555 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-web-config\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015224    2555 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015232    2555 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015240    2555 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d852339b-8569-453f-bdfc-bcf2e3bb69e8-tls-assets\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015248    2555 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015255    2555 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-prometheus-k8s-db\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.015438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.015263    2555 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d852339b-8569-453f-bdfc-bcf2e3bb69e8-config-out\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:00:39.087799 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.087766    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:39.144392 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.144362    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:39.364533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.364505    2555 generic.go:356] "Generic (PLEG): container finished" podID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerID="4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084" exitCode=0
Jul 29 01:00:39.364682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.364564    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084"}
Jul 29 01:00:39.364682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.364589    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.364682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.364633    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d852339b-8569-453f-bdfc-bcf2e3bb69e8","Type":"ContainerDied","Data":"cebbb145d233a25ff17cd636287211bb4c6584d17419d8a14429a4bb95ce2e69"}
Jul 29 01:00:39.364682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.364668    2555 scope.go:117] "RemoveContainer" containerID="40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9"
Jul 29 01:00:39.372255 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.372239    2555 scope.go:117] "RemoveContainer" containerID="9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d"
Jul 29 01:00:39.379307 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.379290    2555 scope.go:117] "RemoveContainer" containerID="4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084"
Jul 29 01:00:39.386100 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.386081    2555 scope.go:117] "RemoveContainer" containerID="979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c"
Jul 29 01:00:39.386740 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.386721    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 01:00:39.390390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.390371    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 01:00:39.392625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.392610    2555 scope.go:117] "RemoveContainer" containerID="48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53"
Jul 29 01:00:39.398628 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.398613    2555 scope.go:117] "RemoveContainer" containerID="7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465"
Jul 29 01:00:39.405474 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.405444    2555 scope.go:117] "RemoveContainer" containerID="e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790"
Jul 29 01:00:39.411717 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.411698    2555 scope.go:117] "RemoveContainer" containerID="40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9"
Jul 29 01:00:39.411950 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.411928    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9\": container with ID starting with 40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9 not found: ID does not exist" containerID="40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9"
Jul 29 01:00:39.412016 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.411957    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9"} err="failed to get container status \"40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9\": rpc error: code = NotFound desc = could not find container \"40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9\": container with ID starting with 40d90717a1a8eb29927ab64401b6d0282c50dd803238ccfef0282e5043f148a9 not found: ID does not exist"
Jul 29 01:00:39.412016 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.411974    2555 scope.go:117] "RemoveContainer" containerID="9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d"
Jul 29 01:00:39.412204 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.412185    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d\": container with ID starting with 9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d not found: ID does not exist" containerID="9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d"
Jul 29 01:00:39.412245 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.412210    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d"} err="failed to get container status \"9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d\": rpc error: code = NotFound desc = could not find container \"9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d\": container with ID starting with 9cb182f6d650cfa9a6ffce65ebb2f83630bfe387703068630df5f1f3ebe3812d not found: ID does not exist"
Jul 29 01:00:39.412245 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.412225    2555 scope.go:117] "RemoveContainer" containerID="4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084"
Jul 29 01:00:39.412550 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.412527    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084\": container with ID starting with 4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084 not found: ID does not exist" containerID="4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084"
Jul 29 01:00:39.412609 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.412561    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084"} err="failed to get container status \"4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084\": rpc error: code = NotFound desc = could not find container \"4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084\": container with ID starting with 4821d4c3c48e2b05f4aea88e7f7554409227cd951696a1e6505b4d2a1d276084 not found: ID does not exist"
Jul 29 01:00:39.412609 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.412581    2555 scope.go:117] "RemoveContainer" containerID="979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c"
Jul 29 01:00:39.412883 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.412859    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c\": container with ID starting with 979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c not found: ID does not exist" containerID="979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c"
Jul 29 01:00:39.413027 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.412889    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c"} err="failed to get container status \"979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c\": rpc error: code = NotFound desc = could not find container \"979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c\": container with ID starting with 979dbb6ad3d7dab04e8856eb4cc1941d783c8c5e0a18f072c1d3b9962e64660c not found: ID does not exist"
Jul 29 01:00:39.413027 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.412912    2555 scope.go:117] "RemoveContainer" containerID="48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53"
Jul 29 01:00:39.413191 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.413167    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53\": container with ID starting with 48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53 not found: ID does not exist" containerID="48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53"
Jul 29 01:00:39.413295 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.413197    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53"} err="failed to get container status \"48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53\": rpc error: code = NotFound desc = could not find container \"48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53\": container with ID starting with 48e82ebf79a3ae75b4fd7bce70e26af8c468483ee35b597a5a7dca9325516b53 not found: ID does not exist"
Jul 29 01:00:39.413295 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.413217    2555 scope.go:117] "RemoveContainer" containerID="7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465"
Jul 29 01:00:39.413514 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.413484    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465\": container with ID starting with 7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465 not found: ID does not exist" containerID="7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465"
Jul 29 01:00:39.413588 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.413515    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465"} err="failed to get container status \"7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465\": rpc error: code = NotFound desc = could not find container \"7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465\": container with ID starting with 7edafa6b38edb490d5403853c0ef26571078aaf9a44247e51f51d3b26b347465 not found: ID does not exist"
Jul 29 01:00:39.413588 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.413535    2555 scope.go:117] "RemoveContainer" containerID="e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790"
Jul 29 01:00:39.413880 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:00:39.413857    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790\": container with ID starting with e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790 not found: ID does not exist" containerID="e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790"
Jul 29 01:00:39.413951 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.413886    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790"} err="failed to get container status \"e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790\": rpc error: code = NotFound desc = could not find container \"e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790\": container with ID starting with e04f986a650ed62b99eca8bc03c212d87e9eb369d97b7a75c503f0dd6a11f790 not found: ID does not exist"
Jul 29 01:00:39.414386 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414370    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 01:00:39.414629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414614    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="thanos-sidecar"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414632    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="thanos-sidecar"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414643    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414651    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414673    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-web"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414682    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-web"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414692    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="init-config-reloader"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414701    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="init-config-reloader"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414711    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="prometheus"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414719    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="prometheus"
Jul 29 01:00:39.414729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414727    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="config-reloader"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414736    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="config-reloader"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414745    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-thanos"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414753    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-thanos"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414845    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-thanos"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414858    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="config-reloader"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414867    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="prometheus"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414874    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414882    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="thanos-sidecar"
Jul 29 01:00:39.415191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.414890    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" containerName="kube-rbac-proxy-web"
Jul 29 01:00:39.419569 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.419552    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.422538 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422515    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 01:00:39.422631 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422593    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 01:00:39.422631 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422519    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-56ssq\""
Jul 29 01:00:39.422631 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422614    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2u6ovpdqgcrk1\""
Jul 29 01:00:39.422786 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422637    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 01:00:39.422786 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422706    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 01:00:39.422883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422789    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 01:00:39.422883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422797    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 01:00:39.422973 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422896    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 01:00:39.422973 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.422966    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 01:00:39.423068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.423027    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 01:00:39.423068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.423059    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 01:00:39.423145 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.423102    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 01:00:39.423358 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.423343    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 01:00:39.426102 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.426052    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 01:00:39.428045 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.428030    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 01:00:39.433020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.432971    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 01:00:39.518623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518551    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518587    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518605    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518660    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518675    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518722    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-web-config\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518743    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518759    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518777    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518796    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88603ea8-5569-4040-afdc-66fd3a79a846-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.518856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518838    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518890    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518951    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518979    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88603ea8-5569-4040-afdc-66fd3a79a846-config-out\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.518998    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.519021    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-config\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.519039    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.519056    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.519071    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.519141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.519103    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltf8p\" (UniqueName: \"kubernetes.io/projected/88603ea8-5569-4040-afdc-66fd3a79a846-kube-api-access-ltf8p\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619496    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619530    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619545    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltf8p\" (UniqueName: \"kubernetes.io/projected/88603ea8-5569-4040-afdc-66fd3a79a846-kube-api-access-ltf8p\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619573    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619591    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619607    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619642    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619659    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619678    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-web-config\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.619997 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.619702    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620092 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620065    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620153 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620115    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620151    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88603ea8-5569-4040-afdc-66fd3a79a846-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620177    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620203    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620238    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620266    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88603ea8-5569-4040-afdc-66fd3a79a846-config-out\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620303    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620312    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620342    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-config\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620368    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.620743 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.620715    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.621106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.621083    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.621666 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.621641    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.622746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.622720    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.622838    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.622851    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.622929    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.623186    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.623357    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.623584    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.623597    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.623889 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.623847    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.624376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.624003    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/88603ea8-5569-4040-afdc-66fd3a79a846-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.624376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.624087    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.625010 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.624988    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-web-config\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.625304 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.625267    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/88603ea8-5569-4040-afdc-66fd3a79a846-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.625649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.625630    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/88603ea8-5569-4040-afdc-66fd3a79a846-config-out\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.625756 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.625740    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/88603ea8-5569-4040-afdc-66fd3a79a846-config\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.627927 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.627910    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltf8p\" (UniqueName: \"kubernetes.io/projected/88603ea8-5569-4040-afdc-66fd3a79a846-kube-api-access-ltf8p\") pod \"prometheus-k8s-0\" (UID: \"88603ea8-5569-4040-afdc-66fd3a79a846\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.729126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.729102    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:39.862894 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:39.862312    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 01:00:40.370366 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:40.370335    2555 generic.go:356] "Generic (PLEG): container finished" podID="88603ea8-5569-4040-afdc-66fd3a79a846" containerID="14b12ff09b7dac6f990c2325364fe3ca4a7aef297584767fae6d058a3d42afec" exitCode=0
Jul 29 01:00:40.370508 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:40.370424    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerDied","Data":"14b12ff09b7dac6f990c2325364fe3ca4a7aef297584767fae6d058a3d42afec"}
Jul 29 01:00:40.370508 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:40.370485    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"0f6d350896f05c5836a06d06439558eed6633da54901962a47c5bc4abe42f2af"}
Jul 29 01:00:40.959835 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:40.959805    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:41.060675 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.060647    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:41.188725 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.188694    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d852339b-8569-453f-bdfc-bcf2e3bb69e8" path="/var/lib/kubelet/pods/d852339b-8569-453f-bdfc-bcf2e3bb69e8/volumes"
Jul 29 01:00:41.377230 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.377140    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"930a8f6b1b1dd492e6bb0773958f6981e1ae63344e94d5cec08841c7dd31fa2d"}
Jul 29 01:00:41.377230 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.377217    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"99ebfd620e47f39dcb481aa62efb7f4a03cb14f6a5a0d2954f6988995d3edf51"}
Jul 29 01:00:41.377458 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.377257    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"e778d90e466c6d98e7ca65c78fdb5f5d977413a35fa86f386abb202885b17b08"}
Jul 29 01:00:41.377458 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.377313    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"5b38c3cdcde7819cbe50e83f4f0b292fa201ddca4fe68172ffa34f2761a5ee83"}
Jul 29 01:00:41.377458 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.377350    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"11c5d5e2c5dffc1bcc8b1552db65792c24df6d1d0227bed760d671a339a27b9c"}
Jul 29 01:00:41.377458 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.377384    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"88603ea8-5569-4040-afdc-66fd3a79a846","Type":"ContainerStarted","Data":"dbd8df8ee3bd8a9b32065ac6333e7ad81cf754a30145504d2ea14763c88e02de"}
Jul 29 01:00:41.406392 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:41.406346    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.382942272 podStartE2EDuration="2.406331854s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.389582ms" isStatefulPod=false podCreationTimestamp="2026-07-29 01:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:00:41.404554472 +0000 UTC m=+370.735204078" watchObservedRunningTime="2026-07-29 01:00:41.406331854 +0000 UTC m=+370.736981460"
Jul 29 01:00:44.729604 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:44.729568    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:00:45.066556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:45.066463    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 01:00:45.068702 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:45.068682    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/034fc553-7c4a-4f22-b467-df36e2e94071-metrics-certs\") pod \"network-metrics-daemon-rb9df\" (UID: \"034fc553-7c4a-4f22-b467-df36e2e94071\") " pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 01:00:45.287251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:45.287222    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bxdbf\""
Jul 29 01:00:45.294578 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:45.294547    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rb9df"
Jul 29 01:00:45.433909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:45.433887    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rb9df"]
Jul 29 01:00:45.436231 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:00:45.436204    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034fc553_7c4a_4f22_b467_df36e2e94071.slice/crio-e390f815725e7a66dc89ede1317638b95eef9fc8aa38f9a727d2852675ff5618 WatchSource:0}: Error finding container e390f815725e7a66dc89ede1317638b95eef9fc8aa38f9a727d2852675ff5618: Status 404 returned error can't find the container with id e390f815725e7a66dc89ede1317638b95eef9fc8aa38f9a727d2852675ff5618
Jul 29 01:00:45.438313 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:45.438299    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:00:46.402358 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:46.402322    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rb9df" event={"ID":"034fc553-7c4a-4f22-b467-df36e2e94071","Type":"ContainerStarted","Data":"e390f815725e7a66dc89ede1317638b95eef9fc8aa38f9a727d2852675ff5618"}
Jul 29 01:00:47.408731 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:47.408696    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rb9df" event={"ID":"034fc553-7c4a-4f22-b467-df36e2e94071","Type":"ContainerStarted","Data":"0e25c3c2034b39e1a58e10b2d9a542027eda7ddc462d5e6af5bcbf605573c4b4"}
Jul 29 01:00:47.409132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:47.408749    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rb9df" event={"ID":"034fc553-7c4a-4f22-b467-df36e2e94071","Type":"ContainerStarted","Data":"521ce28a657f8c410cb98c031a578272ade300ec2c2b89ae382006ab4cbe725e"}
Jul 29 01:00:47.426563 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:47.426485    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rb9df" podStartSLOduration=375.39972027 podStartE2EDuration="6m16.426465639s" totalImagesPullingTime="1.026745369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 00:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:00:47.425567322 +0000 UTC m=+376.756216929" watchObservedRunningTime="2026-07-29 01:00:47.426465639 +0000 UTC m=+376.757115247"
Jul 29 01:00:47.515821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:47.515792    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:47.574041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:47.574007    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:48.005522 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:48.005493    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:48.056646 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:48.056617    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:48.107487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:48.107461    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:55.127214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:55.127179    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:00:55.227522 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:00:55.227492    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:01:34.542697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:34.542659    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:01:34.643389 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:34.643359    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:01:39.729572 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:39.729538    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:01:39.744404 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:39.744377    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:01:40.612600 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:40.612578    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 01:01:41.579849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:41.579816    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:01:41.628731 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:41.628703    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:01:58.410473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:58.410398    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:01:58.460847 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:01:58.460819    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:06.641670 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:06.641628    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:07.401003 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:07.400958    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:10.236529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:10.236499    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:10.336903 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:10.336877    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:14.049250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:14.049215    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:14.636465 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:14.636434    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:14.810068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:14.810035    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:15.866468 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:15.866439    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:16.044354 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:16.044322    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:16.101034 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:16.101007    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:16.442975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:16.442940    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:16.690184 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:16.690156    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:19.629694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:19.629664    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:25.259599 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:25.259567    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:29.507976 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:29.507937    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:29.568121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:29.568089    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:38.144669 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:38.144630    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:38.248147 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:38.248119    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:44.973747 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:44.973712    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:45.076948 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:45.076915    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:47.058625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:47.058590    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:59.360536 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:59.360502    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:02:59.467033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:02:59.466995    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.078625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.078583    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.311357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.311328    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.414847 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.414763    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.457064 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.457029    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"]
Jul 29 01:03:00.460395 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.460372    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.462846 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.462811    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 01:03:00.463002 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.462901    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 01:03:00.463002 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.462961    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 01:03:00.463108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.463039    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-88xjl\""
Jul 29 01:03:00.463108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.463061    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 01:03:00.489456 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.489412    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"]
Jul 29 01:03:00.598717 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.598671    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.622091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.622062    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77d5b793-e5e3-4bf6-9571-ba6949991084-manifests\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.622091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.622091    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d5b793-e5e3-4bf6-9571-ba6949991084-tmp\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.622338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.622122    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77d5b793-e5e3-4bf6-9571-ba6949991084-webhook-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.622338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.622205    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nmns\" (UniqueName: \"kubernetes.io/projected/77d5b793-e5e3-4bf6-9571-ba6949991084-kube-api-access-2nmns\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.622338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.622248    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77d5b793-e5e3-4bf6-9571-ba6949991084-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.723450 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.723417    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nmns\" (UniqueName: \"kubernetes.io/projected/77d5b793-e5e3-4bf6-9571-ba6949991084-kube-api-access-2nmns\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.723638 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.723458    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77d5b793-e5e3-4bf6-9571-ba6949991084-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.723638 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.723510    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77d5b793-e5e3-4bf6-9571-ba6949991084-manifests\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.723638 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.723529    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d5b793-e5e3-4bf6-9571-ba6949991084-tmp\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.723638 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.723566    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77d5b793-e5e3-4bf6-9571-ba6949991084-webhook-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.723989 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.723952    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/77d5b793-e5e3-4bf6-9571-ba6949991084-manifests\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.725816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.725795    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77d5b793-e5e3-4bf6-9571-ba6949991084-tmp\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.726087 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.726067    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/77d5b793-e5e3-4bf6-9571-ba6949991084-webhook-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.726168 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.726132    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/77d5b793-e5e3-4bf6-9571-ba6949991084-apiservice-cert\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.731928 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.731910    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nmns\" (UniqueName: \"kubernetes.io/projected/77d5b793-e5e3-4bf6-9571-ba6949991084-kube-api-access-2nmns\") pod \"opendatahub-operator-controller-manager-795cff746b-jf2w9\" (UID: \"77d5b793-e5e3-4bf6-9571-ba6949991084\") " pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.752962 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.752922    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.770928 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.770902    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:00.854254 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.854217    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:00.924104 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.924055    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"]
Jul 29 01:03:00.927812 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:00.927784    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d5b793_e5e3_4bf6_9571_ba6949991084.slice/crio-eed153af8f68497965cbb265c2b097c312352a1747b7aa14e46a2ca9395fddc9 WatchSource:0}: Error finding container eed153af8f68497965cbb265c2b097c312352a1747b7aa14e46a2ca9395fddc9: Status 404 returned error can't find the container with id eed153af8f68497965cbb265c2b097c312352a1747b7aa14e46a2ca9395fddc9
Jul 29 01:03:00.936910 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:00.936890    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:01.005733 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:01.005667    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:01.882519 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:01.882479    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9" event={"ID":"77d5b793-e5e3-4bf6-9571-ba6949991084","Type":"ContainerStarted","Data":"eed153af8f68497965cbb265c2b097c312352a1747b7aa14e46a2ca9395fddc9"}
Jul 29 01:03:04.896365 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:04.896329    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9" event={"ID":"77d5b793-e5e3-4bf6-9571-ba6949991084","Type":"ContainerStarted","Data":"8f90da429ff1005f7ec5cc467b044aa90b37f648a7851286260ed27569c5433e"}
Jul 29 01:03:05.902429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:05.902394    2555 generic.go:356] "Generic (PLEG): container finished" podID="77d5b793-e5e3-4bf6-9571-ba6949991084" containerID="8f90da429ff1005f7ec5cc467b044aa90b37f648a7851286260ed27569c5433e" exitCode=0
Jul 29 01:03:05.902854 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:05.902505    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9" event={"ID":"77d5b793-e5e3-4bf6-9571-ba6949991084","Type":"ContainerDied","Data":"8f90da429ff1005f7ec5cc467b044aa90b37f648a7851286260ed27569c5433e"}
Jul 29 01:03:06.907924 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:06.907859    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9" event={"ID":"77d5b793-e5e3-4bf6-9571-ba6949991084","Type":"ContainerStarted","Data":"ca9ad3c2648514c066d13b1020697ece03ead9649d64e892646a036f76aa199d"}
Jul 29 01:03:06.908338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:06.907964    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:06.932289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:06.932221    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9" podStartSLOduration=2.189130194 podStartE2EDuration="6.932203371s" totalImagesPullingTime="3.731051482s" totalInitContainerRuntime="1.012021695s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:06.930213293 +0000 UTC m=+516.260862924" watchObservedRunningTime="2026-07-29 01:03:06.932203371 +0000 UTC m=+516.262852977"
Jul 29 01:03:17.914190 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:17.914161    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-795cff746b-jf2w9"
Jul 29 01:03:20.802596 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:20.802563    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:20.901650 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:20.901623    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:21.224132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.224095    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:21.366825 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.366794    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"]
Jul 29 01:03:21.376164 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.376140    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.379075 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.379047    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 01:03:21.379181 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.379106    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-769kg\""
Jul 29 01:03:21.380495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.380469    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 01:03:21.380611 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.380543    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 01:03:21.380611 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.380563    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 01:03:21.383991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.383967    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"]
Jul 29 01:03:21.393400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.393367    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8258603-0360-4bd2-95b2-e7474d23d308-tls-certs\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.393523 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.393419    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8258603-0360-4bd2-95b2-e7474d23d308-tmp\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.393523 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.393470    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk4vv\" (UniqueName: \"kubernetes.io/projected/f8258603-0360-4bd2-95b2-e7474d23d308-kube-api-access-kk4vv\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.465623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.465599    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:21.494097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.494032    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8258603-0360-4bd2-95b2-e7474d23d308-tls-certs\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.494097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.494063    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8258603-0360-4bd2-95b2-e7474d23d308-tmp\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.494097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.494084    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk4vv\" (UniqueName: \"kubernetes.io/projected/f8258603-0360-4bd2-95b2-e7474d23d308-kube-api-access-kk4vv\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.496322 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.496304    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8258603-0360-4bd2-95b2-e7474d23d308-tmp\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.496539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.496522    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8258603-0360-4bd2-95b2-e7474d23d308-tls-certs\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.506747 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.506726    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk4vv\" (UniqueName: \"kubernetes.io/projected/f8258603-0360-4bd2-95b2-e7474d23d308-kube-api-access-kk4vv\") pod \"kube-auth-proxy-7b44fbcd-7p9fx\" (UID: \"f8258603-0360-4bd2-95b2-e7474d23d308\") " pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.598503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.598471    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h7xtb"]
Jul 29 01:03:21.601306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.601285    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:21.605479 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.605439    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 01:03:21.605651 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.605624    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-72hh5\""
Jul 29 01:03:21.614095 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.614069    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h7xtb"]
Jul 29 01:03:21.632912 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.632837    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:21.687205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.687177    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"
Jul 29 01:03:21.695787 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.695758    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01814704-c87c-4bbd-9b47-9d51634bf7fb-cert\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:21.695903 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.695818    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h52j7\" (UniqueName: \"kubernetes.io/projected/01814704-c87c-4bbd-9b47-9d51634bf7fb-kube-api-access-h52j7\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:21.729151 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.729125    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:21.797002 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.796969    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01814704-c87c-4bbd-9b47-9d51634bf7fb-cert\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:21.797188 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.797051    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h52j7\" (UniqueName: \"kubernetes.io/projected/01814704-c87c-4bbd-9b47-9d51634bf7fb-kube-api-access-h52j7\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:21.797188 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:21.797165    2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 29 01:03:21.797338 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:21.797245    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01814704-c87c-4bbd-9b47-9d51634bf7fb-cert podName:01814704-c87c-4bbd-9b47-9d51634bf7fb nodeName:}" failed. No retries permitted until 2026-07-29 01:03:22.297220903 +0000 UTC m=+531.627870488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/01814704-c87c-4bbd-9b47-9d51634bf7fb-cert") pod "odh-model-controller-7998c67dc4-h7xtb" (UID: "01814704-c87c-4bbd-9b47-9d51634bf7fb") : secret "odh-model-controller-webhook-cert" not found
Jul 29 01:03:21.817689 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.817659    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h52j7\" (UniqueName: \"kubernetes.io/projected/01814704-c87c-4bbd-9b47-9d51634bf7fb-kube-api-access-h52j7\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:21.836691 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.836656    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx"]
Jul 29 01:03:21.841131 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:21.841084    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8258603_0360_4bd2_95b2_e7474d23d308.slice/crio-fc63bd3a73f8c7b5662f9b040ebc222326a66a61a34c21f1388b433b52cf0a16 WatchSource:0}: Error finding container fc63bd3a73f8c7b5662f9b040ebc222326a66a61a34c21f1388b433b52cf0a16: Status 404 returned error can't find the container with id fc63bd3a73f8c7b5662f9b040ebc222326a66a61a34c21f1388b433b52cf0a16
Jul 29 01:03:21.848105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.848081    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:21.966810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:21.966760    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx" event={"ID":"f8258603-0360-4bd2-95b2-e7474d23d308","Type":"ContainerStarted","Data":"fc63bd3a73f8c7b5662f9b040ebc222326a66a61a34c21f1388b433b52cf0a16"}
Jul 29 01:03:22.224891 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.224859    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:22.300296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.300245    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01814704-c87c-4bbd-9b47-9d51634bf7fb-cert\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:22.302690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.302662    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01814704-c87c-4bbd-9b47-9d51634bf7fb-cert\") pod \"odh-model-controller-7998c67dc4-h7xtb\" (UID: \"01814704-c87c-4bbd-9b47-9d51634bf7fb\") " pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:22.324222 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.324194    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jjw7d"]
Jul 29 01:03:22.327669 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.327649    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:22.331644 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.331626    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-l58qs\""
Jul 29 01:03:22.331805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.331782    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 01:03:22.344501 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.344480    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jjw7d"]
Jul 29 01:03:22.370574 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.370548    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:22.401180 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.401155    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c84fa1-9280-4c6c-9836-a25983dc6586-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:22.401355 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.401199    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpg4k\" (UniqueName: \"kubernetes.io/projected/88c84fa1-9280-4c6c-9836-a25983dc6586-kube-api-access-xpg4k\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:22.501838 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.501760    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpg4k\" (UniqueName: \"kubernetes.io/projected/88c84fa1-9280-4c6c-9836-a25983dc6586-kube-api-access-xpg4k\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:22.501838 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.501838    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c84fa1-9280-4c6c-9836-a25983dc6586-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:22.502018 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:22.501938    2555 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 29 01:03:22.502018 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:22.501988    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c84fa1-9280-4c6c-9836-a25983dc6586-tls-certs podName:88c84fa1-9280-4c6c-9836-a25983dc6586 nodeName:}" failed. No retries permitted until 2026-07-29 01:03:23.001972505 +0000 UTC m=+532.332622090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/88c84fa1-9280-4c6c-9836-a25983dc6586-tls-certs") pod "model-serving-api-54d89bd5fc-jjw7d" (UID: "88c84fa1-9280-4c6c-9836-a25983dc6586") : secret "model-serving-api-tls" not found
Jul 29 01:03:22.511155 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.511126    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpg4k\" (UniqueName: \"kubernetes.io/projected/88c84fa1-9280-4c6c-9836-a25983dc6586-kube-api-access-xpg4k\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:22.512999 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.512974    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:22.660129 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.660099    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-h7xtb"]
Jul 29 01:03:22.663512 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:22.662821    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01814704_c87c_4bbd_9b47_9d51634bf7fb.slice/crio-d674d81a1f73b028950fc40023fe2bef4aa3df6fa889428d9633574e86383a8f WatchSource:0}: Error finding container d674d81a1f73b028950fc40023fe2bef4aa3df6fa889428d9633574e86383a8f: Status 404 returned error can't find the container with id d674d81a1f73b028950fc40023fe2bef4aa3df6fa889428d9633574e86383a8f
Jul 29 01:03:22.726657 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.726627    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:22.794205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.794127    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"]
Jul 29 01:03:22.797268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.797246    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.797902 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.797874    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:22.800041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.800015    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 01:03:22.800041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.800035    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 01:03:22.800189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.800047    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 01:03:22.800189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.800055    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 01:03:22.800189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.800057    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7ftf7\""
Jul 29 01:03:22.811270 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.811248    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"]
Jul 29 01:03:22.855960 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.855915    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:22.906139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906106    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.906313 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906148    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.906313 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906178    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.906313 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906242    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.906473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906379    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.906473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906406    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgwfx\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-kube-api-access-cgwfx\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.906473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.906432    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5616c0fd-a346-4cfe-a861-572e5bc662d2-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:22.973432 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:22.973393    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb" event={"ID":"01814704-c87c-4bbd-9b47-9d51634bf7fb","Type":"ContainerStarted","Data":"d674d81a1f73b028950fc40023fe2bef4aa3df6fa889428d9633574e86383a8f"}
Jul 29 01:03:23.006993 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.006961    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.006993 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.006994    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.007198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007030    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.007198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007069    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.007198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007144    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c84fa1-9280-4c6c-9836-a25983dc6586-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:23.007384 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007202    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.007384 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007225    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgwfx\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-kube-api-access-cgwfx\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.007384 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007255    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5616c0fd-a346-4cfe-a861-572e5bc662d2-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.007752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.007724    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.009979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.009954    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5616c0fd-a346-4cfe-a861-572e5bc662d2-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.010087 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.010064    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.010196 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.010174    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.010196 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.010188    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.010561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.010544    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c84fa1-9280-4c6c-9836-a25983dc6586-tls-certs\") pod \"model-serving-api-54d89bd5fc-jjw7d\" (UID: \"88c84fa1-9280-4c6c-9836-a25983dc6586\") " pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:23.015482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.015458    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.015778 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.015750    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgwfx\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-kube-api-access-cgwfx\") pod \"istiod-openshift-gateway-7b7995d5cd-wx8sn\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.107859 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.107779    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:23.240130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.240060    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:23.271158 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.271081    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"]
Jul 29 01:03:23.275411 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:23.275380    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5616c0fd_a346_4cfe_a861_572e5bc662d2.slice/crio-7ef7a2ffc7303da807d68c61775e205f6e30bd7aa2e92c46a93849a68a8892d5 WatchSource:0}: Error finding container 7ef7a2ffc7303da807d68c61775e205f6e30bd7aa2e92c46a93849a68a8892d5: Status 404 returned error can't find the container with id 7ef7a2ffc7303da807d68c61775e205f6e30bd7aa2e92c46a93849a68a8892d5
Jul 29 01:03:23.402908 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.402791    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jjw7d"]
Jul 29 01:03:23.406566 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:23.406521    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c84fa1_9280_4c6c_9836_a25983dc6586.slice/crio-af995f13972070a58d962b5749701f3618ac1740f155a4c164738728e95eaa47 WatchSource:0}: Error finding container af995f13972070a58d962b5749701f3618ac1740f155a4c164738728e95eaa47: Status 404 returned error can't find the container with id af995f13972070a58d962b5749701f3618ac1740f155a4c164738728e95eaa47
Jul 29 01:03:23.773859 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.773811    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:23.981436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.981332    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" event={"ID":"5616c0fd-a346-4cfe-a861-572e5bc662d2","Type":"ContainerStarted","Data":"7ef7a2ffc7303da807d68c61775e205f6e30bd7aa2e92c46a93849a68a8892d5"}
Jul 29 01:03:23.983154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:23.983094    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d" event={"ID":"88c84fa1-9280-4c6c-9836-a25983dc6586","Type":"ContainerStarted","Data":"af995f13972070a58d962b5749701f3618ac1740f155a4c164738728e95eaa47"}
Jul 29 01:03:27.000524 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.000455    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx" event={"ID":"f8258603-0360-4bd2-95b2-e7474d23d308","Type":"ContainerStarted","Data":"61460fd010d7136e618f03597d3729cd1a6a88940d583e17424b21ee13948e41"}
Jul 29 01:03:27.002113 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.002074    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb" event={"ID":"01814704-c87c-4bbd-9b47-9d51634bf7fb","Type":"ContainerStarted","Data":"076945c133d7a89f8d762a334618443546858ae076e27d3dc560649c8f69ffbe"}
Jul 29 01:03:27.002248 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.002134    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:27.003794 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.003760    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d" event={"ID":"88c84fa1-9280-4c6c-9836-a25983dc6586","Type":"ContainerStarted","Data":"1fb0470b07e14bc8862309a133c961747fcc352f606f6b4543477f853af0ec16"}
Jul 29 01:03:27.003915 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.003887    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:27.017758 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.017716    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b44fbcd-7p9fx" podStartSLOduration=1.352831034 podStartE2EDuration="6.017703835s" totalImagesPullingTime="4.664872801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:27.016536744 +0000 UTC m=+536.347186351" watchObservedRunningTime="2026-07-29 01:03:27.017703835 +0000 UTC m=+536.348353440"
Jul 29 01:03:27.033048 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.032993    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb" podStartSLOduration=2.125560129 podStartE2EDuration="6.032978656s" totalImagesPullingTime="3.907418527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:27.032064792 +0000 UTC m=+536.362714399" watchObservedRunningTime="2026-07-29 01:03:27.032978656 +0000 UTC m=+536.363628261"
Jul 29 01:03:27.049894 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:27.049010    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d" podStartSLOduration=1.95136584 podStartE2EDuration="5.048988139s" totalImagesPullingTime="3.097622299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:27.047728832 +0000 UTC m=+536.378378438" watchObservedRunningTime="2026-07-29 01:03:27.048988139 +0000 UTC m=+536.379637745"
Jul 29 01:03:28.386823 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:28.386773    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:03:28.387091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:28.386856    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:03:29.014717 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.014682    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" event={"ID":"5616c0fd-a346-4cfe-a861-572e5bc662d2","Type":"ContainerStarted","Data":"b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb"}
Jul 29 01:03:29.014956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.014939    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:29.016419 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.016386    2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-wx8sn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 01:03:29.016555 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.016442    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" podUID="5616c0fd-a346-4cfe-a861-572e5bc662d2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 01:03:29.038844 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.038800    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" podStartSLOduration=1.930284727 podStartE2EDuration="7.038788467s" totalImagesPullingTime="5.10850374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:29.036743511 +0000 UTC m=+538.367393119" watchObservedRunningTime="2026-07-29 01:03:29.038788467 +0000 UTC m=+538.369438073"
Jul 29 01:03:29.204078 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.204041    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:29.314338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.314235    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:29.494707 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.494674    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:29.656244 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:29.656157    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:30.019806 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:30.019778    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:30.073303 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:30.072633    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:32.640688 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.640652    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:32.695813 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.695775    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"]
Jul 29 01:03:32.700753 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.700734    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"]
Jul 29 01:03:32.700876 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.700858    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:32.703725 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.703704    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 01:03:32.703725 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.703717    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sjph4\""
Jul 29 01:03:32.703876 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.703800    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.706940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.706898    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 01:03:32.707062 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.706972    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lfbmf\""
Jul 29 01:03:32.711574 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.711550    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"]
Jul 29 01:03:32.713634 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.713605    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"]
Jul 29 01:03:32.731095 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.731065    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:32.806790 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.806762    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6hmn\" (UniqueName: \"kubernetes.io/projected/54dc7351-13ba-44e3-81e9-e2af92bd6c6c-kube-api-access-l6hmn\") pod \"llmisvc-controller-manager-777c6bd7d5-d7k7x\" (UID: \"54dc7351-13ba-44e3-81e9-e2af92bd6c6c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.806911 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.806828    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f97c401-139c-4bba-b887-372be01cad11-cert\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:32.806911 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.806852    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7z5r\" (UniqueName: \"kubernetes.io/projected/2f97c401-139c-4bba-b887-372be01cad11-kube-api-access-k7z5r\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:32.806911 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.806870    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54dc7351-13ba-44e3-81e9-e2af92bd6c6c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d7k7x\" (UID: \"54dc7351-13ba-44e3-81e9-e2af92bd6c6c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.844356 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.844333    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:32.907663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.907603    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6hmn\" (UniqueName: \"kubernetes.io/projected/54dc7351-13ba-44e3-81e9-e2af92bd6c6c-kube-api-access-l6hmn\") pod \"llmisvc-controller-manager-777c6bd7d5-d7k7x\" (UID: \"54dc7351-13ba-44e3-81e9-e2af92bd6c6c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.907787 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.907669    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f97c401-139c-4bba-b887-372be01cad11-cert\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:32.907787 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.907702    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7z5r\" (UniqueName: \"kubernetes.io/projected/2f97c401-139c-4bba-b887-372be01cad11-kube-api-access-k7z5r\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:32.907787 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.907738    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54dc7351-13ba-44e3-81e9-e2af92bd6c6c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d7k7x\" (UID: \"54dc7351-13ba-44e3-81e9-e2af92bd6c6c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.907949 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:32.907826    2555 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 29 01:03:32.907949 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:32.907899    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f97c401-139c-4bba-b887-372be01cad11-cert podName:2f97c401-139c-4bba-b887-372be01cad11 nodeName:}" failed. No retries permitted until 2026-07-29 01:03:33.407878417 +0000 UTC m=+542.738528019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2f97c401-139c-4bba-b887-372be01cad11-cert") pod "kserve-controller-manager-d6f4b65b8-jngrx" (UID: "2f97c401-139c-4bba-b887-372be01cad11") : secret "kserve-webhook-server-cert" not found
Jul 29 01:03:32.910297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.910251    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54dc7351-13ba-44e3-81e9-e2af92bd6c6c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d7k7x\" (UID: \"54dc7351-13ba-44e3-81e9-e2af92bd6c6c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.929189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.929166    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6hmn\" (UniqueName: \"kubernetes.io/projected/54dc7351-13ba-44e3-81e9-e2af92bd6c6c-kube-api-access-l6hmn\") pod \"llmisvc-controller-manager-777c6bd7d5-d7k7x\" (UID: \"54dc7351-13ba-44e3-81e9-e2af92bd6c6c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:32.932088 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.932067    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7z5r\" (UniqueName: \"kubernetes.io/projected/2f97c401-139c-4bba-b887-372be01cad11-kube-api-access-k7z5r\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:32.980106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:32.980085    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:33.019403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.019374    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:33.073294 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.073247    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:33.148161 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.148132    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"]
Jul 29 01:03:33.150828 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.150808    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:33.151324 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:33.151304    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54dc7351_13ba_44e3_81e9_e2af92bd6c6c.slice/crio-0cbb43866ed7074d39a8662f477606b13c26c9dc7922e93354d496e7085920a5 WatchSource:0}: Error finding container 0cbb43866ed7074d39a8662f477606b13c26c9dc7922e93354d496e7085920a5: Status 404 returned error can't find the container with id 0cbb43866ed7074d39a8662f477606b13c26c9dc7922e93354d496e7085920a5
Jul 29 01:03:33.411093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.411056    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f97c401-139c-4bba-b887-372be01cad11-cert\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:33.413304 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.413262    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f97c401-139c-4bba-b887-372be01cad11-cert\") pod \"kserve-controller-manager-d6f4b65b8-jngrx\" (UID: \"2f97c401-139c-4bba-b887-372be01cad11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:33.612677 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.612635    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:33.753661 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:33.753634    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"]
Jul 29 01:03:33.755463 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:03:33.755435    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f97c401_139c_4bba_b887_372be01cad11.slice/crio-5892f0c17d485550f4dfbaf322e5673f5ad2a4ea6b6dcd2099426ad09739c8de WatchSource:0}: Error finding container 5892f0c17d485550f4dfbaf322e5673f5ad2a4ea6b6dcd2099426ad09739c8de: Status 404 returned error can't find the container with id 5892f0c17d485550f4dfbaf322e5673f5ad2a4ea6b6dcd2099426ad09739c8de
Jul 29 01:03:34.035941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:34.035904    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx" event={"ID":"2f97c401-139c-4bba-b887-372be01cad11","Type":"ContainerStarted","Data":"5892f0c17d485550f4dfbaf322e5673f5ad2a4ea6b6dcd2099426ad09739c8de"}
Jul 29 01:03:34.036885 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:34.036861    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x" event={"ID":"54dc7351-13ba-44e3-81e9-e2af92bd6c6c","Type":"ContainerStarted","Data":"0cbb43866ed7074d39a8662f477606b13c26c9dc7922e93354d496e7085920a5"}
Jul 29 01:03:36.497525 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.497487    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"]
Jul 29 01:03:36.497880 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.497735    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" podUID="5616c0fd-a346-4cfe-a861-572e5bc662d2" containerName="discovery" containerID="cri-o://b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb" gracePeriod=30
Jul 29 01:03:36.774300 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.774259    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:36.845126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845099    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-kubeconfig\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845132    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-cacerts\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845340 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845173    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-ca-configmap\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845340 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845207    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-token\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845340 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845238    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-dns-cert\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845340 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845301    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5616c0fd-a346-4cfe-a861-572e5bc662d2-local-certs\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845546 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845343    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgwfx\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-kube-api-access-cgwfx\") pod \"5616c0fd-a346-4cfe-a861-572e5bc662d2\" (UID: \"5616c0fd-a346-4cfe-a861-572e5bc662d2\") "
Jul 29 01:03:36.845644 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.845599    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:03:36.847696 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.847667    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:03:36.847824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.847804    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5616c0fd-a346-4cfe-a861-572e5bc662d2-local-certs" (OuterVolumeSpecName: "local-certs") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:03:36.847909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.847885    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:03:36.847959 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.847908    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:36.848012 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.847983    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-cacerts" (OuterVolumeSpecName: "cacerts") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:03:36.848085 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.848068    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-kube-api-access-cgwfx" (OuterVolumeSpecName: "kube-api-access-cgwfx") pod "5616c0fd-a346-4cfe-a861-572e5bc662d2" (UID: "5616c0fd-a346-4cfe-a861-572e5bc662d2"). InnerVolumeSpecName "kube-api-access-cgwfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:03:36.946668 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946631    2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-ca-configmap\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:36.946668 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946663    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:36.946862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946673    2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-csr-dns-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:36.946862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946695    2555 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5616c0fd-a346-4cfe-a861-572e5bc662d2-local-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:36.946862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946704    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgwfx\" (UniqueName: \"kubernetes.io/projected/5616c0fd-a346-4cfe-a861-572e5bc662d2-kube-api-access-cgwfx\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:36.946862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946712    2555 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-istio-kubeconfig\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:36.946862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:36.946722    2555 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5616c0fd-a346-4cfe-a861-572e5bc662d2-cacerts\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:03:37.051192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.051110    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx" event={"ID":"2f97c401-139c-4bba-b887-372be01cad11","Type":"ContainerStarted","Data":"98053b6af350090e884997698aadcc4c2c1d5a651899b2a968a32ff4faab683d"}
Jul 29 01:03:37.051370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.051213    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:03:37.052402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.052375    2555 generic.go:356] "Generic (PLEG): container finished" podID="5616c0fd-a346-4cfe-a861-572e5bc662d2" containerID="b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb" exitCode=0
Jul 29 01:03:37.052497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.052414    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"
Jul 29 01:03:37.052497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.052447    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" event={"ID":"5616c0fd-a346-4cfe-a861-572e5bc662d2","Type":"ContainerDied","Data":"b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb"}
Jul 29 01:03:37.052497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.052483    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn" event={"ID":"5616c0fd-a346-4cfe-a861-572e5bc662d2","Type":"ContainerDied","Data":"7ef7a2ffc7303da807d68c61775e205f6e30bd7aa2e92c46a93849a68a8892d5"}
Jul 29 01:03:37.052602 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.052509    2555 scope.go:117] "RemoveContainer" containerID="b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb"
Jul 29 01:03:37.062709 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.062628    2555 scope.go:117] "RemoveContainer" containerID="b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb"
Jul 29 01:03:37.062998 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:03:37.062972    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb\": container with ID starting with b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb not found: ID does not exist" containerID="b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb"
Jul 29 01:03:37.063093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.063008    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb"} err="failed to get container status \"b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb\": rpc error: code = NotFound desc = could not find container \"b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb\": container with ID starting with b0bd7f8ecd251623ae37e80aa0b66a80ac6030a73c5425e6bd31b8b538c4a2eb not found: ID does not exist"
Jul 29 01:03:37.068910 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.068849    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx" podStartSLOduration=2.767922555 podStartE2EDuration="5.068835952s" totalImagesPullingTime="2.300913397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:37.06842997 +0000 UTC m=+546.399079576" watchObservedRunningTime="2026-07-29 01:03:37.068835952 +0000 UTC m=+546.399485559"
Jul 29 01:03:37.083152 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.083130    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"]
Jul 29 01:03:37.086920 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.086901    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-wx8sn"]
Jul 29 01:03:37.188422 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:37.188388    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5616c0fd-a346-4cfe-a861-572e5bc662d2" path="/var/lib/kubelet/pods/5616c0fd-a346-4cfe-a861-572e5bc662d2/volumes"
Jul 29 01:03:38.011062 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:38.011041    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-h7xtb"
Jul 29 01:03:38.012836 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:38.012818    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jjw7d"
Jul 29 01:03:38.060858 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:38.060824    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x" event={"ID":"54dc7351-13ba-44e3-81e9-e2af92bd6c6c","Type":"ContainerStarted","Data":"9ea08d5da3c41836f20e7ba1f89009df0f72c54bc9feed20238d65748a369785"}
Jul 29 01:03:38.078429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:38.078382    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x" podStartSLOduration=1.278292658 podStartE2EDuration="6.078368289s" totalImagesPullingTime="4.800075631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:03:38.076931456 +0000 UTC m=+547.407581066" watchObservedRunningTime="2026-07-29 01:03:38.078368289 +0000 UTC m=+547.409017894"
Jul 29 01:03:39.065247 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:39.065219    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:03:40.898580 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:40.898543    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:03:40.949803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:03:40.949769    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:08.066355 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:08.066316    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jngrx"
Jul 29 01:04:09.822894 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:09.822827    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:09.878401 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:09.878371    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:10.072702 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:10.072672    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d7k7x"
Jul 29 01:04:11.622555 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:11.622520    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:11.682127 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:11.682095    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:12.779207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:12.779172    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:22.255599 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.255562    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:22.387443 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.387411    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7wjdb"]
Jul 29 01:04:22.387751 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.387735    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5616c0fd-a346-4cfe-a861-572e5bc662d2" containerName="discovery"
Jul 29 01:04:22.387856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.387753    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5616c0fd-a346-4cfe-a861-572e5bc662d2" containerName="discovery"
Jul 29 01:04:22.387856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.387823    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5616c0fd-a346-4cfe-a861-572e5bc662d2" containerName="discovery"
Jul 29 01:04:22.395710 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.395685    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:22.399007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.398977    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hzslz\""
Jul 29 01:04:22.399231 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.399203    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7wjdb"]
Jul 29 01:04:22.400462 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.400439    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 01:04:22.400767 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.400720    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 01:04:22.526674 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.526582    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmsq7\" (UniqueName: \"kubernetes.io/projected/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf-kube-api-access-zmsq7\") pod \"kuadrant-operator-catalog-7wjdb\" (UID: \"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf\") " pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:22.627718 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.627682    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmsq7\" (UniqueName: \"kubernetes.io/projected/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf-kube-api-access-zmsq7\") pod \"kuadrant-operator-catalog-7wjdb\" (UID: \"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf\") " pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:22.640017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.639985    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmsq7\" (UniqueName: \"kubernetes.io/projected/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf-kube-api-access-zmsq7\") pod \"kuadrant-operator-catalog-7wjdb\" (UID: \"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf\") " pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:22.707085 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.707051    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:22.750862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.750825    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7wjdb"]
Jul 29 01:04:22.834914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.834766    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7wjdb"]
Jul 29 01:04:22.837890 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:04:22.837860    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a07314a_f34e_4cdd_a0da_4d0a4df53fcf.slice/crio-edfeea08061b47ded46669fc9b20a31cf582c798d7d6a48172a31bfc26631683 WatchSource:0}: Error finding container edfeea08061b47ded46669fc9b20a31cf582c798d7d6a48172a31bfc26631683: Status 404 returned error can't find the container with id edfeea08061b47ded46669fc9b20a31cf582c798d7d6a48172a31bfc26631683
Jul 29 01:04:22.961729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.961688    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s8hlq"]
Jul 29 01:04:22.964397 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.964380    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:22.972855 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:22.972827    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s8hlq"]
Jul 29 01:04:23.031719 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:23.031691    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmpzs\" (UniqueName: \"kubernetes.io/projected/0272b4f0-97aa-4f0a-baf4-b5b533feddb9-kube-api-access-wmpzs\") pod \"kuadrant-operator-catalog-s8hlq\" (UID: \"0272b4f0-97aa-4f0a-baf4-b5b533feddb9\") " pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:23.132766 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:23.132678    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmpzs\" (UniqueName: \"kubernetes.io/projected/0272b4f0-97aa-4f0a-baf4-b5b533feddb9-kube-api-access-wmpzs\") pod \"kuadrant-operator-catalog-s8hlq\" (UID: \"0272b4f0-97aa-4f0a-baf4-b5b533feddb9\") " pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:23.141633 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:23.141608    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmpzs\" (UniqueName: \"kubernetes.io/projected/0272b4f0-97aa-4f0a-baf4-b5b533feddb9-kube-api-access-wmpzs\") pod \"kuadrant-operator-catalog-s8hlq\" (UID: \"0272b4f0-97aa-4f0a-baf4-b5b533feddb9\") " pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:23.247165 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:23.247133    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb" event={"ID":"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf","Type":"ContainerStarted","Data":"edfeea08061b47ded46669fc9b20a31cf582c798d7d6a48172a31bfc26631683"}
Jul 29 01:04:23.274261 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:23.274223    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:23.408982 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:23.407101    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s8hlq"]
Jul 29 01:04:23.414804 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:04:23.414777    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0272b4f0_97aa_4f0a_baf4_b5b533feddb9.slice/crio-774ca2260323c887c1cca5be91643da290d7cabc49916cdc6b5c0ce84d9cc9f0 WatchSource:0}: Error finding container 774ca2260323c887c1cca5be91643da290d7cabc49916cdc6b5c0ce84d9cc9f0: Status 404 returned error can't find the container with id 774ca2260323c887c1cca5be91643da290d7cabc49916cdc6b5c0ce84d9cc9f0
Jul 29 01:04:24.254966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:24.254927    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq" event={"ID":"0272b4f0-97aa-4f0a-baf4-b5b533feddb9","Type":"ContainerStarted","Data":"774ca2260323c887c1cca5be91643da290d7cabc49916cdc6b5c0ce84d9cc9f0"}
Jul 29 01:04:24.745900 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:24.745869    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:25.261863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.261829    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq" event={"ID":"0272b4f0-97aa-4f0a-baf4-b5b533feddb9","Type":"ContainerStarted","Data":"85b19d31d79d278271cf7f52ce701286a338e92b99aa98b03ae60aede91a4bc3"}
Jul 29 01:04:25.263378 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.263354    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb" event={"ID":"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf","Type":"ContainerStarted","Data":"1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec"}
Jul 29 01:04:25.263504 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.263469    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb" podUID="6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" containerName="registry-server" containerID="cri-o://1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec" gracePeriod=2
Jul 29 01:04:25.279436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.279381    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq" podStartSLOduration=1.5680354909999998 podStartE2EDuration="3.27936606s" totalImagesPullingTime="1.711330569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:25.278672104 +0000 UTC m=+594.609321711" watchObservedRunningTime="2026-07-29 01:04:25.27936606 +0000 UTC m=+594.610015697"
Jul 29 01:04:25.296017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.295901    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb" podStartSLOduration=1.004194751 podStartE2EDuration="3.295882261s" totalImagesPullingTime="2.29168751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:25.295469479 +0000 UTC m=+594.626119103" watchObservedRunningTime="2026-07-29 01:04:25.295882261 +0000 UTC m=+594.626531868"
Jul 29 01:04:25.505537 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.505514    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7wjdb_6a07314a-f34e-4cdd-a0da-4d0a4df53fcf/registry-server/0.log"
Jul 29 01:04:25.507844 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.507827    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:25.656805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.656721    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmsq7\" (UniqueName: \"kubernetes.io/projected/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf-kube-api-access-zmsq7\") pod \"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf\" (UID: \"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf\") "
Jul 29 01:04:25.658852 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.658826    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf-kube-api-access-zmsq7" (OuterVolumeSpecName: "kube-api-access-zmsq7") pod "6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" (UID: "6a07314a-f34e-4cdd-a0da-4d0a4df53fcf"). InnerVolumeSpecName "kube-api-access-zmsq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:04:25.757594 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:25.757562    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmsq7\" (UniqueName: \"kubernetes.io/projected/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf-kube-api-access-zmsq7\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:04:26.269233 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.269205    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-7wjdb_6a07314a-f34e-4cdd-a0da-4d0a4df53fcf/registry-server/0.log"
Jul 29 01:04:26.269429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.269242    2555 generic.go:356] "Generic (PLEG): container finished" podID="6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" containerID="1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec" exitCode=1
Jul 29 01:04:26.269429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.269271    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb" event={"ID":"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf","Type":"ContainerDied","Data":"1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec"}
Jul 29 01:04:26.269429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.269319    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb"
Jul 29 01:04:26.269429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.269337    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7wjdb" event={"ID":"6a07314a-f34e-4cdd-a0da-4d0a4df53fcf","Type":"ContainerDied","Data":"edfeea08061b47ded46669fc9b20a31cf582c798d7d6a48172a31bfc26631683"}
Jul 29 01:04:26.269429 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.269358    2555 scope.go:117] "RemoveContainer" containerID="1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec"
Jul 29 01:04:26.278958 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.278940    2555 scope.go:117] "RemoveContainer" containerID="1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec"
Jul 29 01:04:26.279187 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:04:26.279170    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec\": container with ID starting with 1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec not found: ID does not exist" containerID="1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec"
Jul 29 01:04:26.279229 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.279196    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec"} err="failed to get container status \"1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec\": rpc error: code = NotFound desc = could not find container \"1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec\": container with ID starting with 1f88b83673254fede90c8f55cf910010102f24740a53c67d22dbbb6743dd83ec not found: ID does not exist"
Jul 29 01:04:26.290402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.290371    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7wjdb"]
Jul 29 01:04:26.293561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:26.293536    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7wjdb"]
Jul 29 01:04:27.189419 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:27.189377    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" path="/var/lib/kubelet/pods/6a07314a-f34e-4cdd-a0da-4d0a4df53fcf/volumes"
Jul 29 01:04:33.274506 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:33.274477    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:33.276780 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:33.274558    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:33.295616 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:33.295593    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:33.318931 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:33.318911    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-s8hlq"
Jul 29 01:04:39.514618 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:39.514584    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:39.574284 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:39.574257    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:39.680099 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:39.680075    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:40.264053 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:40.264019    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:48.501655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:48.501624    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:50.115935 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:50.115900    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:50.176381 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:50.176349    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:50.233515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:50.233487    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:51.122289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.122252    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"]
Jul 29 01:04:51.122667 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.122629    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" containerName="registry-server"
Jul 29 01:04:51.122667 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.122642    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" containerName="registry-server"
Jul 29 01:04:51.122736 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.122707    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a07314a-f34e-4cdd-a0da-4d0a4df53fcf" containerName="registry-server"
Jul 29 01:04:51.125793 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.125774    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:04:51.128874 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.128852    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-n59d7\""
Jul 29 01:04:51.129037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.129019    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 29 01:04:51.135796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.135775    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"]
Jul 29 01:04:51.273426 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.273395    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg7ff\" (UniqueName: \"kubernetes.io/projected/0fc625e9-1b6e-443c-b0ee-9844adbdcd1b-kube-api-access-gg7ff\") pod \"dns-operator-controller-manager-864dd78c4c-h6ls4\" (UID: \"0fc625e9-1b6e-443c-b0ee-9844adbdcd1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:04:51.373905 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.373826    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg7ff\" (UniqueName: \"kubernetes.io/projected/0fc625e9-1b6e-443c-b0ee-9844adbdcd1b-kube-api-access-gg7ff\") pod \"dns-operator-controller-manager-864dd78c4c-h6ls4\" (UID: \"0fc625e9-1b6e-443c-b0ee-9844adbdcd1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:04:51.384473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.384448    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg7ff\" (UniqueName: \"kubernetes.io/projected/0fc625e9-1b6e-443c-b0ee-9844adbdcd1b-kube-api-access-gg7ff\") pod \"dns-operator-controller-manager-864dd78c4c-h6ls4\" (UID: \"0fc625e9-1b6e-443c-b0ee-9844adbdcd1b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:04:51.436253 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.436226    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:04:51.569338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:51.569312    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"]
Jul 29 01:04:51.571343 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:04:51.571311    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc625e9_1b6e_443c_b0ee_9844adbdcd1b.slice/crio-0ed4798caf6be38e7bb08b8c0ebdb3c928bcdc77c0d4187adad476f014aeb146 WatchSource:0}: Error finding container 0ed4798caf6be38e7bb08b8c0ebdb3c928bcdc77c0d4187adad476f014aeb146: Status 404 returned error can't find the container with id 0ed4798caf6be38e7bb08b8c0ebdb3c928bcdc77c0d4187adad476f014aeb146
Jul 29 01:04:52.379751 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:52.379714    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4" event={"ID":"0fc625e9-1b6e-443c-b0ee-9844adbdcd1b","Type":"ContainerStarted","Data":"0ed4798caf6be38e7bb08b8c0ebdb3c928bcdc77c0d4187adad476f014aeb146"}
Jul 29 01:04:52.692801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:52.692777    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:54.559985 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.559961    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"]
Jul 29 01:04:54.563065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.563049    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:04:54.566404 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.566382    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fdkjn\""
Jul 29 01:04:54.578414 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.578393    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"]
Jul 29 01:04:54.597509 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.597482    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwf6f\" (UniqueName: \"kubernetes.io/projected/5d1f68da-5efd-45ae-9040-feea9fcf9eb6-kube-api-access-zwf6f\") pod \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" (UID: \"5d1f68da-5efd-45ae-9040-feea9fcf9eb6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:04:54.698493 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.698456    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwf6f\" (UniqueName: \"kubernetes.io/projected/5d1f68da-5efd-45ae-9040-feea9fcf9eb6-kube-api-access-zwf6f\") pod \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" (UID: \"5d1f68da-5efd-45ae-9040-feea9fcf9eb6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:04:54.712221 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.712186    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwf6f\" (UniqueName: \"kubernetes.io/projected/5d1f68da-5efd-45ae-9040-feea9fcf9eb6-kube-api-access-zwf6f\") pod \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" (UID: \"5d1f68da-5efd-45ae-9040-feea9fcf9eb6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:04:54.872776 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:54.872673    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:04:55.041636 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.041593    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"]
Jul 29 01:04:55.045463 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:04:55.045432    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d1f68da_5efd_45ae_9040_feea9fcf9eb6.slice/crio-9c880786adb5c6534cc95b86ded8d27d72de9c7809d2a1999679e6f8455b9a87 WatchSource:0}: Error finding container 9c880786adb5c6534cc95b86ded8d27d72de9c7809d2a1999679e6f8455b9a87: Status 404 returned error can't find the container with id 9c880786adb5c6534cc95b86ded8d27d72de9c7809d2a1999679e6f8455b9a87
Jul 29 01:04:55.166828 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.166738    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:55.280157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.280124    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:55.398205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.398169    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4" event={"ID":"0fc625e9-1b6e-443c-b0ee-9844adbdcd1b","Type":"ContainerStarted","Data":"55934382f0ebdf975e9fc00ef6e88facd7e687cbe8251a59748cc022c5ba809a"}
Jul 29 01:04:55.398409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.398268    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:04:55.399469 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.399444    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9" event={"ID":"5d1f68da-5efd-45ae-9040-feea9fcf9eb6","Type":"ContainerStarted","Data":"9c880786adb5c6534cc95b86ded8d27d72de9c7809d2a1999679e6f8455b9a87"}
Jul 29 01:04:55.421482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:55.421402    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4" podStartSLOduration=1.484791126 podStartE2EDuration="4.421390149s" totalImagesPullingTime="2.936599023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:55.4206623 +0000 UTC m=+624.751311906" watchObservedRunningTime="2026-07-29 01:04:55.421390149 +0000 UTC m=+624.752039755"
Jul 29 01:04:56.512663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:56.512609    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:57.411019 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:57.410983    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9" event={"ID":"5d1f68da-5efd-45ae-9040-feea9fcf9eb6","Type":"ContainerStarted","Data":"0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7"}
Jul 29 01:04:57.411205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:57.411111    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:04:57.446154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:57.446104    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9" podStartSLOduration=1.61305651 podStartE2EDuration="3.446091966s" totalImagesPullingTime="1.833035456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:04:57.443297653 +0000 UTC m=+626.773947251" watchObservedRunningTime="2026-07-29 01:04:57.446091966 +0000 UTC m=+626.776741572"
Jul 29 01:04:59.420806 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.420775    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"]
Jul 29 01:04:59.429061 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.429035    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.431818 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.431797    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-75x59\""
Jul 29 01:04:59.434327 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.434271    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wc4t\" (UniqueName: \"kubernetes.io/projected/cd24f282-cb58-4b39-b0b6-7e94c0d64798-kube-api-access-2wc4t\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.434436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.434368    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd24f282-cb58-4b39-b0b6-7e94c0d64798-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.439168 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.439146    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"]
Jul 29 01:04:59.535400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.535365    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wc4t\" (UniqueName: \"kubernetes.io/projected/cd24f282-cb58-4b39-b0b6-7e94c0d64798-kube-api-access-2wc4t\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.535552 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.535425    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd24f282-cb58-4b39-b0b6-7e94c0d64798-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.535798 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.535780    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd24f282-cb58-4b39-b0b6-7e94c0d64798-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.544134 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.544105    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wc4t\" (UniqueName: \"kubernetes.io/projected/cd24f282-cb58-4b39-b0b6-7e94c0d64798-kube-api-access-2wc4t\") pod \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.696536 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.696508    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:04:59.738972 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.738942    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:04:59.876934 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:04:59.876903    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"]
Jul 29 01:04:59.880554 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:04:59.880529    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd24f282_cb58_4b39_b0b6_7e94c0d64798.slice/crio-a98cb6a9163b59e232791fe7592fe4bcf7534108f076b06bcfca446e200da91f WatchSource:0}: Error finding container a98cb6a9163b59e232791fe7592fe4bcf7534108f076b06bcfca446e200da91f: Status 404 returned error can't find the container with id a98cb6a9163b59e232791fe7592fe4bcf7534108f076b06bcfca446e200da91f
Jul 29 01:05:00.086810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:00.086724    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:00.314542 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:00.314510    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:00.413833 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:00.413759    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:00.424304 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:00.424267    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm" event={"ID":"cd24f282-cb58-4b39-b0b6-7e94c0d64798","Type":"ContainerStarted","Data":"a98cb6a9163b59e232791fe7592fe4bcf7534108f076b06bcfca446e200da91f"}
Jul 29 01:05:02.748052 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.748018    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"]
Jul 29 01:05:02.751637 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.751615    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:02.754431 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.754412    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vgngj\""
Jul 29 01:05:02.763048 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.763026    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"]
Jul 29 01:05:02.763318 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.763294    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnqsk\" (UniqueName: \"kubernetes.io/projected/ce1cce01-bb9f-468d-b956-a81197604262-kube-api-access-qnqsk\") pod \"authorino-operator-75cc8d5cb7-mm9wj\" (UID: \"ce1cce01-bb9f-468d-b956-a81197604262\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:02.864035 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.863990    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnqsk\" (UniqueName: \"kubernetes.io/projected/ce1cce01-bb9f-468d-b956-a81197604262-kube-api-access-qnqsk\") pod \"authorino-operator-75cc8d5cb7-mm9wj\" (UID: \"ce1cce01-bb9f-468d-b956-a81197604262\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:02.873782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:02.873751    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnqsk\" (UniqueName: \"kubernetes.io/projected/ce1cce01-bb9f-468d-b956-a81197604262-kube-api-access-qnqsk\") pod \"authorino-operator-75cc8d5cb7-mm9wj\" (UID: \"ce1cce01-bb9f-468d-b956-a81197604262\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:03.062594 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:03.062504    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:03.128438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:03.128400    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:03.199927 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:03.199904    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"]
Jul 29 01:05:03.202325 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:05:03.202298    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce1cce01_bb9f_468d_b956_a81197604262.slice/crio-903e93cf758607726a7fbdc8c62ce945c970650fa5b085411c4cb8b205841048 WatchSource:0}: Error finding container 903e93cf758607726a7fbdc8c62ce945c970650fa5b085411c4cb8b205841048: Status 404 returned error can't find the container with id 903e93cf758607726a7fbdc8c62ce945c970650fa5b085411c4cb8b205841048
Jul 29 01:05:03.442780 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:03.442747    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj" event={"ID":"ce1cce01-bb9f-468d-b956-a81197604262","Type":"ContainerStarted","Data":"903e93cf758607726a7fbdc8c62ce945c970650fa5b085411c4cb8b205841048"}
Jul 29 01:05:04.491171 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:04.491075    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:04.591521 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:04.591425    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:06.406853 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:06.406813    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h6ls4"
Jul 29 01:05:07.466191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:07.466158    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm" event={"ID":"cd24f282-cb58-4b39-b0b6-7e94c0d64798","Type":"ContainerStarted","Data":"d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61"}
Jul 29 01:05:07.466678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:07.466258    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:07.467782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:07.467758    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj" event={"ID":"ce1cce01-bb9f-468d-b956-a81197604262","Type":"ContainerStarted","Data":"d780cece7be0ab0bf806b1d2d18722787b1c03ce106336be637ffc35a582adbe"}
Jul 29 01:05:07.467933 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:07.467920    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:07.489207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:07.489154    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm" podStartSLOduration=1.923979429 podStartE2EDuration="8.48913552s" totalImagesPullingTime="6.565156091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:07.486859206 +0000 UTC m=+636.817508813" watchObservedRunningTime="2026-07-29 01:05:07.48913552 +0000 UTC m=+636.819785128"
Jul 29 01:05:07.507208 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:07.507166    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj" podStartSLOduration=2.228981472 podStartE2EDuration="5.507158061s" totalImagesPullingTime="3.278176589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:07.506513526 +0000 UTC m=+636.837163131" watchObservedRunningTime="2026-07-29 01:05:07.507158061 +0000 UTC m=+636.837807666"
Jul 29 01:05:08.286436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:08.286402    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:08.386554 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:08.386519    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:08.418432 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:08.418403    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:09.385319 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:09.385265    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:09.680895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:09.680798    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:11.685802 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:11.685770    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:11.743639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:11.743606    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:18.475668 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:18.475633    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:18.476045 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:18.475785    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mm9wj"
Jul 29 01:05:19.476141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.476105    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"]
Jul 29 01:05:19.479694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.479677    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:19.497689 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.497665    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"]
Jul 29 01:05:19.596174 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.596134    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"]
Jul 29 01:05:19.596465 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:19.596439    2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-c49cw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac"
Jul 29 01:05:19.605916 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.605890    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:19.606014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.605947    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49cw\" (UniqueName: \"kubernetes.io/projected/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-kube-api-access-c49cw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:19.706672 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.706642    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:19.706823 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.706690    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49cw\" (UniqueName: \"kubernetes.io/projected/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-kube-api-access-c49cw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:19.707056 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.707033    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:19.718156 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:19.718130    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49cw\" (UniqueName: \"kubernetes.io/projected/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-kube-api-access-c49cw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:20.063117 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.063081    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"]
Jul 29 01:05:20.070154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.070122    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"]
Jul 29 01:05:20.083004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.082977    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"]
Jul 29 01:05:20.083180 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.083143    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" containerName="manager" containerID="cri-o://d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61" gracePeriod=2
Jul 29 01:05:20.085807 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.085760    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.088233 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.088209    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"]
Jul 29 01:05:20.100837 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.100812    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"]
Jul 29 01:05:20.101039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.101017    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" containerName="manager" containerID="cri-o://0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7" gracePeriod=2
Jul 29 01:05:20.103411 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.103387    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.112814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.112252    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"]
Jul 29 01:05:20.114185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114161    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"]
Jul 29 01:05:20.114701 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114676    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" containerName="manager"
Jul 29 01:05:20.114801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114702    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" containerName="manager"
Jul 29 01:05:20.114801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114739    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" containerName="manager"
Jul 29 01:05:20.114801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114748    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" containerName="manager"
Jul 29 01:05:20.114976 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114837    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" containerName="manager"
Jul 29 01:05:20.114976 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.114850    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" containerName="manager"
Jul 29 01:05:20.118097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.118077    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.120611 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.120577    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.122682 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.122651    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.130004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.129977    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"]
Jul 29 01:05:20.134160 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.134136    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"]
Jul 29 01:05:20.137221 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.137202    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:20.144350 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.144328    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:20.152723 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.152699    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"]
Jul 29 01:05:20.181767 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.181739    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.184120 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.184095    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.214162 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.214134    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d73a756-a17c-4e94-baa3-6323dace3230-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hhbt9\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.214265 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.214182    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zbvr\" (UniqueName: \"kubernetes.io/projected/b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7-kube-api-access-6zbvr\") pod \"limitador-operator-controller-manager-859d7b78bd-7vhrt\" (UID: \"b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:20.214340 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.214271    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b29b4\" (UniqueName: \"kubernetes.io/projected/0d73a756-a17c-4e94-baa3-6323dace3230-kube-api-access-b29b4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hhbt9\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.286039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.285968    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:20.315323 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.315234    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d73a756-a17c-4e94-baa3-6323dace3230-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hhbt9\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.315323 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.315295    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zbvr\" (UniqueName: \"kubernetes.io/projected/b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7-kube-api-access-6zbvr\") pod \"limitador-operator-controller-manager-859d7b78bd-7vhrt\" (UID: \"b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:20.315529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.315338    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b29b4\" (UniqueName: \"kubernetes.io/projected/0d73a756-a17c-4e94-baa3-6323dace3230-kube-api-access-b29b4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hhbt9\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.315697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.315667    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d73a756-a17c-4e94-baa3-6323dace3230-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hhbt9\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.324121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.324100    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zbvr\" (UniqueName: \"kubernetes.io/projected/b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7-kube-api-access-6zbvr\") pod \"limitador-operator-controller-manager-859d7b78bd-7vhrt\" (UID: \"b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:20.324359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.324339    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b29b4\" (UniqueName: \"kubernetes.io/projected/0d73a756-a17c-4e94-baa3-6323dace3230-kube-api-access-b29b4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hhbt9\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.339515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.339492    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.341502 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.341485    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.342054 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.342028    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.344306 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.344269    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.346356 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.346337    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.348345 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.348328    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.416466 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.416443    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wc4t\" (UniqueName: \"kubernetes.io/projected/cd24f282-cb58-4b39-b0b6-7e94c0d64798-kube-api-access-2wc4t\") pod \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") "
Jul 29 01:05:20.416564 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.416527    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd24f282-cb58-4b39-b0b6-7e94c0d64798-extensions-socket-volume\") pod \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\" (UID: \"cd24f282-cb58-4b39-b0b6-7e94c0d64798\") "
Jul 29 01:05:20.416627 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.416567    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwf6f\" (UniqueName: \"kubernetes.io/projected/5d1f68da-5efd-45ae-9040-feea9fcf9eb6-kube-api-access-zwf6f\") pod \"5d1f68da-5efd-45ae-9040-feea9fcf9eb6\" (UID: \"5d1f68da-5efd-45ae-9040-feea9fcf9eb6\") "
Jul 29 01:05:20.416942 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.416920    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd24f282-cb58-4b39-b0b6-7e94c0d64798-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cd24f282-cb58-4b39-b0b6-7e94c0d64798" (UID: "cd24f282-cb58-4b39-b0b6-7e94c0d64798"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:05:20.418556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.418531    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd24f282-cb58-4b39-b0b6-7e94c0d64798-kube-api-access-2wc4t" (OuterVolumeSpecName: "kube-api-access-2wc4t") pod "cd24f282-cb58-4b39-b0b6-7e94c0d64798" (UID: "cd24f282-cb58-4b39-b0b6-7e94c0d64798"). InnerVolumeSpecName "kube-api-access-2wc4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:05:20.418736 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.418712    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d1f68da-5efd-45ae-9040-feea9fcf9eb6-kube-api-access-zwf6f" (OuterVolumeSpecName: "kube-api-access-zwf6f") pod "5d1f68da-5efd-45ae-9040-feea9fcf9eb6" (UID: "5d1f68da-5efd-45ae-9040-feea9fcf9eb6"). InnerVolumeSpecName "kube-api-access-zwf6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:05:20.517352 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.517328    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wc4t\" (UniqueName: \"kubernetes.io/projected/cd24f282-cb58-4b39-b0b6-7e94c0d64798-kube-api-access-2wc4t\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:20.517352 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.517348    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd24f282-cb58-4b39-b0b6-7e94c0d64798-extensions-socket-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:20.517670 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.517358    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwf6f\" (UniqueName: \"kubernetes.io/projected/5d1f68da-5efd-45ae-9040-feea9fcf9eb6-kube-api-access-zwf6f\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:20.525776 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.525758    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:20.526956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.526935    2555 generic.go:356] "Generic (PLEG): container finished" podID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" containerID="0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7" exitCode=0
Jul 29 01:05:20.527026 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.526977    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.527079 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.526980    2555 scope.go:117] "RemoveContainer" containerID="0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7"
Jul 29 01:05:20.528604 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.528428    2555 generic.go:356] "Generic (PLEG): container finished" podID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" containerID="d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61" exitCode=0
Jul 29 01:05:20.528604 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.528477    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.528713 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.528670    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:20.530969 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.530941    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.531459 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.531439    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:20.533220 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.533191    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.535534 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.535515    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.537629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.537307    2555 scope.go:117] "RemoveContainer" containerID="0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7"
Jul 29 01:05:20.537629 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.537587    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7\": container with ID starting with 0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7 not found: ID does not exist" containerID="0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7"
Jul 29 01:05:20.537629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.537608    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7"} err="failed to get container status \"0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7\": rpc error: code = NotFound desc = could not find container \"0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7\": container with ID starting with 0a79bd8736014a2f3b814b2002c9d46109a6b07f6c60b31f086702698f0b43b7 not found: ID does not exist"
Jul 29 01:05:20.537629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.537624    2555 scope.go:117] "RemoveContainer" containerID="d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61"
Jul 29 01:05:20.538207 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.537727    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.538207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.537880    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:20.540130 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.539981    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:20.542235 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.542205    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:20.544463 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.544432    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.546578 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.546485    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.546671 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.546618    2555 scope.go:117] "RemoveContainer" containerID="d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61"
Jul 29 01:05:20.546931 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.546911    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61\": container with ID starting with d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61 not found: ID does not exist" containerID="d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61"
Jul 29 01:05:20.546989 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.546941    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61"} err="failed to get container status \"d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61\": rpc error: code = NotFound desc = could not find container \"d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61\": container with ID starting with d74274c9e2c681e652ca4bd5e883fbbc198a762540a84f36ad9a7335629f1e61 not found: ID does not exist"
Jul 29 01:05:20.548690 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.548670    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:20.550556 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.550532    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:20.552599 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:20.552568    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:20.617916 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.617882    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-extensions-socket-volume\") pod \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") "
Jul 29 01:05:20.618084 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.617952    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c49cw\" (UniqueName: \"kubernetes.io/projected/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-kube-api-access-c49cw\") pod \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\" (UID: \"ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac\") "
Jul 29 01:05:20.618287 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.618243    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" (UID: "ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:05:20.620079 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.620047    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-kube-api-access-c49cw" (OuterVolumeSpecName: "kube-api-access-c49cw") pod "ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" (UID: "ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac"). InnerVolumeSpecName "kube-api-access-c49cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:05:20.680139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.680110    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"]
Jul 29 01:05:20.682964 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:05:20.682935    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d73a756_a17c_4e94_baa3_6323dace3230.slice/crio-81b77f6345989a460350408246b70917dc31e4f59fccabaf6114050c93db77c5 WatchSource:0}: Error finding container 81b77f6345989a460350408246b70917dc31e4f59fccabaf6114050c93db77c5: Status 404 returned error can't find the container with id 81b77f6345989a460350408246b70917dc31e4f59fccabaf6114050c93db77c5
Jul 29 01:05:20.693754 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.693723    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"]
Jul 29 01:05:20.696009 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:05:20.695986    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7eaf57c_d80c_46fc_bc3f_7a61cc409bc7.slice/crio-cab15d412a02c10202cab7ec5864e8b8712be6ebebfc5efc2acc6947d18f5e73 WatchSource:0}: Error finding container cab15d412a02c10202cab7ec5864e8b8712be6ebebfc5efc2acc6947d18f5e73: Status 404 returned error can't find the container with id cab15d412a02c10202cab7ec5864e8b8712be6ebebfc5efc2acc6947d18f5e73
Jul 29 01:05:20.719068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.719046    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c49cw\" (UniqueName: \"kubernetes.io/projected/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-kube-api-access-c49cw\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:20.719068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:20.719069    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac-extensions-socket-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:21.188609 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:21.188574    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8wnx9\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8wnx9"
Jul 29 01:05:21.190071 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.190049    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d1f68da-5efd-45ae-9040-feea9fcf9eb6" path="/var/lib/kubelet/pods/5d1f68da-5efd-45ae-9040-feea9fcf9eb6/volumes"
Jul 29 01:05:21.190421 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.190408    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" path="/var/lib/kubelet/pods/cd24f282-cb58-4b39-b0b6-7e94c0d64798/volumes"
Jul 29 01:05:21.190724 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.190711    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" path="/var/lib/kubelet/pods/ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac/volumes"
Jul 29 01:05:21.190907 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:21.190888    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-qb2wm\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="cd24f282-cb58-4b39-b0b6-7e94c0d64798" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-qb2wm"
Jul 29 01:05:21.193221 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:21.193202    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:21.535417 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.535330    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt" event={"ID":"b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7","Type":"ContainerStarted","Data":"dfc96d1760d19672808c745a694e68b4842e3584c9cf7f92255863ffa93b2936"}
Jul 29 01:05:21.535901 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.535657    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt" event={"ID":"b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7","Type":"ContainerStarted","Data":"cab15d412a02c10202cab7ec5864e8b8712be6ebebfc5efc2acc6947d18f5e73"}
Jul 29 01:05:21.535901 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.535700    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:21.538906 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.538880    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9" event={"ID":"0d73a756-a17c-4e94-baa3-6323dace3230","Type":"ContainerStarted","Data":"58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315"}
Jul 29 01:05:21.538906 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.538903    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:21.539067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.538926    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9" event={"ID":"0d73a756-a17c-4e94-baa3-6323dace3230","Type":"ContainerStarted","Data":"81b77f6345989a460350408246b70917dc31e4f59fccabaf6114050c93db77c5"}
Jul 29 01:05:21.539067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.539022    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:21.555144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.555102    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt" podStartSLOduration=1.555089553 podStartE2EDuration="1.555089553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:21.553460474 +0000 UTC m=+650.884110081" watchObservedRunningTime="2026-07-29 01:05:21.555089553 +0000 UTC m=+650.885739229"
Jul 29 01:05:21.555537 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:21.555514    2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-5vqnr\" is forbidden: User \"system:node:ip-10-0-139-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-6.ec2.internal' and this object" podUID="ea1eb2f4-bf5f-4026-88b4-1d1892ac8eac" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5vqnr"
Jul 29 01:05:21.580913 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:21.580874    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9" podStartSLOduration=1.580863059 podStartE2EDuration="1.580863059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:21.579955127 +0000 UTC m=+650.910604736" watchObservedRunningTime="2026-07-29 01:05:21.580863059 +0000 UTC m=+650.911512706"
Jul 29 01:05:22.756613 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:22.756578    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:22.860151 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:22.860121    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:25.134271 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:25.134234    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:25.218936 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:25.218909    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:25.561875 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:25.561839    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:25.663459 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:25.663432    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:29.187043 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:29.187005    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:29.256732 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:29.256703    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:32.545710 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:32.545674    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7vhrt"
Jul 29 01:05:32.546191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:32.545760    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:38.021082 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:38.021045    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:38.229024 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:38.228987    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:40.635305 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:40.635258    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:40.730789 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:40.730749    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:43.793861 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:43.793828    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:43.894021 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:43.893992    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:47.315608 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.315566    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"]
Jul 29 01:05:47.316071 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.315791    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9" podUID="0d73a756-a17c-4e94-baa3-6323dace3230" containerName="manager" containerID="cri-o://58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315" gracePeriod=10
Jul 29 01:05:47.563312 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.563271    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:47.650182 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.650118    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d73a756-a17c-4e94-baa3-6323dace3230-extensions-socket-volume\") pod \"0d73a756-a17c-4e94-baa3-6323dace3230\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") "
Jul 29 01:05:47.650182 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.650171    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b29b4\" (UniqueName: \"kubernetes.io/projected/0d73a756-a17c-4e94-baa3-6323dace3230-kube-api-access-b29b4\") pod \"0d73a756-a17c-4e94-baa3-6323dace3230\" (UID: \"0d73a756-a17c-4e94-baa3-6323dace3230\") "
Jul 29 01:05:47.650577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.650550    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d73a756-a17c-4e94-baa3-6323dace3230-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0d73a756-a17c-4e94-baa3-6323dace3230" (UID: "0d73a756-a17c-4e94-baa3-6323dace3230"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:05:47.652096 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.652071    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d73a756-a17c-4e94-baa3-6323dace3230-kube-api-access-b29b4" (OuterVolumeSpecName: "kube-api-access-b29b4") pod "0d73a756-a17c-4e94-baa3-6323dace3230" (UID: "0d73a756-a17c-4e94-baa3-6323dace3230"). InnerVolumeSpecName "kube-api-access-b29b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:05:47.662408 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.662389    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:47.667320 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.667298    2555 generic.go:356] "Generic (PLEG): container finished" podID="0d73a756-a17c-4e94-baa3-6323dace3230" containerID="58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315" exitCode=0
Jul 29 01:05:47.667390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.667337    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9" event={"ID":"0d73a756-a17c-4e94-baa3-6323dace3230","Type":"ContainerDied","Data":"58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315"}
Jul 29 01:05:47.667390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.667346    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"
Jul 29 01:05:47.667390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.667379    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9" event={"ID":"0d73a756-a17c-4e94-baa3-6323dace3230","Type":"ContainerDied","Data":"81b77f6345989a460350408246b70917dc31e4f59fccabaf6114050c93db77c5"}
Jul 29 01:05:47.667493 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.667401    2555 scope.go:117] "RemoveContainer" containerID="58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315"
Jul 29 01:05:47.675682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.675662    2555 scope.go:117] "RemoveContainer" containerID="58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315"
Jul 29 01:05:47.675947 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:05:47.675918    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315\": container with ID starting with 58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315 not found: ID does not exist" containerID="58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315"
Jul 29 01:05:47.676051 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.675949    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315"} err="failed to get container status \"58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315\": rpc error: code = NotFound desc = could not find container \"58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315\": container with ID starting with 58d60d437a5e0b71c56d362f02e1fe117d62bb034159a61f9a8ca81cf405e315 not found: ID does not exist"
Jul 29 01:05:47.712029 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.712004    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"]
Jul 29 01:05:47.723255 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.723229    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hhbt9"]
Jul 29 01:05:47.747684 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.747662    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"]
Jul 29 01:05:47.747974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.747963    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d73a756-a17c-4e94-baa3-6323dace3230" containerName="manager"
Jul 29 01:05:47.748017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.747976    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d73a756-a17c-4e94-baa3-6323dace3230" containerName="manager"
Jul 29 01:05:47.748048 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.748041    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d73a756-a17c-4e94-baa3-6323dace3230" containerName="manager"
Jul 29 01:05:47.750453 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.750438    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:47.750640 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.750620    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d73a756-a17c-4e94-baa3-6323dace3230-extensions-socket-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:47.750696 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.750642    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b29b4\" (UniqueName: \"kubernetes.io/projected/0d73a756-a17c-4e94-baa3-6323dace3230-kube-api-access-b29b4\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:05:47.754011 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.753993    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-75x59\""
Jul 29 01:05:47.773443 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.773423    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"]
Jul 29 01:05:47.791026 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.790987    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:47.851105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.851082    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/000eebd0-2113-4a1b-96ff-97e13428a192-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvhs2\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:47.851190 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.851143    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lktdt\" (UniqueName: \"kubernetes.io/projected/000eebd0-2113-4a1b-96ff-97e13428a192-kube-api-access-lktdt\") pod \"kuadrant-operator-controller-manager-86878c769f-nvhs2\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:47.951984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.951953    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/000eebd0-2113-4a1b-96ff-97e13428a192-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvhs2\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:47.952117 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.952019    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lktdt\" (UniqueName: \"kubernetes.io/projected/000eebd0-2113-4a1b-96ff-97e13428a192-kube-api-access-lktdt\") pod \"kuadrant-operator-controller-manager-86878c769f-nvhs2\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:47.952351 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.952333    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/000eebd0-2113-4a1b-96ff-97e13428a192-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nvhs2\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:47.968065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:47.968043    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lktdt\" (UniqueName: \"kubernetes.io/projected/000eebd0-2113-4a1b-96ff-97e13428a192-kube-api-access-lktdt\") pod \"kuadrant-operator-controller-manager-86878c769f-nvhs2\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:48.060390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:48.060365    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:48.208622 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:48.208553    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"]
Jul 29 01:05:48.212125 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:05:48.212100    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000eebd0_2113_4a1b_96ff_97e13428a192.slice/crio-e2f1263b1d269a4d1265b03292a8cb9160b3dcaf5ccecdbed0b04c58646ade34 WatchSource:0}: Error finding container e2f1263b1d269a4d1265b03292a8cb9160b3dcaf5ccecdbed0b04c58646ade34: Status 404 returned error can't find the container with id e2f1263b1d269a4d1265b03292a8cb9160b3dcaf5ccecdbed0b04c58646ade34
Jul 29 01:05:48.674246 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:48.674167    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2" event={"ID":"000eebd0-2113-4a1b-96ff-97e13428a192","Type":"ContainerStarted","Data":"efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745"}
Jul 29 01:05:48.674246 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:48.674219    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2" event={"ID":"000eebd0-2113-4a1b-96ff-97e13428a192","Type":"ContainerStarted","Data":"e2f1263b1d269a4d1265b03292a8cb9160b3dcaf5ccecdbed0b04c58646ade34"}
Jul 29 01:05:48.674734 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:48.674298    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:49.190196 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:49.190161    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d73a756-a17c-4e94-baa3-6323dace3230" path="/var/lib/kubelet/pods/0d73a756-a17c-4e94-baa3-6323dace3230/volumes"
Jul 29 01:05:57.379971 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:57.379922    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:57.480339 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:57.480306    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:57.981878 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:57.981838    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:58.054522 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:58.054489    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:05:59.682749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:59.682715    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:05:59.707070 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:05:59.707014    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2" podStartSLOduration=12.706998939 podStartE2EDuration="12.706998939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:05:48.702162647 +0000 UTC m=+678.032812255" watchObservedRunningTime="2026-07-29 01:05:59.706998939 +0000 UTC m=+689.037648620"
Jul 29 01:06:03.401427 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:03.401388    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:03.500592 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:03.500557    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:03.763357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:03.763328    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:03.995390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:03.995355    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:06.487775 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.487738    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:06.488215 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.488142    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:06.491098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.491082    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.493744 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.493720    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 01:06:06.493874 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.493770    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-r7kqd\""
Jul 29 01:06:06.503128 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.503107    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:06.591308 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.591259    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:06.597807 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.597781    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953c7e2e-5e2b-447d-a82a-dbe02d314571-config-file\") pod \"limitador-limitador-64c94fd64f-hqw2z\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.597947 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.597841    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrss\" (UniqueName: \"kubernetes.io/projected/953c7e2e-5e2b-447d-a82a-dbe02d314571-kube-api-access-khrss\") pod \"limitador-limitador-64c94fd64f-hqw2z\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.602575 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.602553    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:06.699099 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.699061    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953c7e2e-5e2b-447d-a82a-dbe02d314571-config-file\") pod \"limitador-limitador-64c94fd64f-hqw2z\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.699262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.699141    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrss\" (UniqueName: \"kubernetes.io/projected/953c7e2e-5e2b-447d-a82a-dbe02d314571-kube-api-access-khrss\") pod \"limitador-limitador-64c94fd64f-hqw2z\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.699710 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.699691    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953c7e2e-5e2b-447d-a82a-dbe02d314571-config-file\") pod \"limitador-limitador-64c94fd64f-hqw2z\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.704710 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.704688    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:06.708394 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.708375    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrss\" (UniqueName: \"kubernetes.io/projected/953c7e2e-5e2b-447d-a82a-dbe02d314571-kube-api-access-khrss\") pod \"limitador-limitador-64c94fd64f-hqw2z\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.801181 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.801097    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:06.934741 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.934715    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:06.936594 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:06.936567    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953c7e2e_5e2b_447d_a82a_dbe02d314571.slice/crio-6a3b2172257310b7fcc2f74abbb2dfdfa8d50ff743b4c3ab042b18b8a4534abc WatchSource:0}: Error finding container 6a3b2172257310b7fcc2f74abbb2dfdfa8d50ff743b4c3ab042b18b8a4534abc: Status 404 returned error can't find the container with id 6a3b2172257310b7fcc2f74abbb2dfdfa8d50ff743b4c3ab042b18b8a4534abc
Jul 29 01:06:06.938514 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:06.938491    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:06:07.330860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.330830    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:07.402979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.402947    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:07.526337 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.526295    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz82m"]
Jul 29 01:06:07.530834 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.530816    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:07.533853 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.533833    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dtfm8\""
Jul 29 01:06:07.538158 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.538133    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz82m"]
Jul 29 01:06:07.608904 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.608835    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcz9h\" (UniqueName: \"kubernetes.io/projected/c0b1d023-aa4c-4e1a-b682-5580e9204727-kube-api-access-hcz9h\") pod \"authorino-75d556bd77-hz82m\" (UID: \"c0b1d023-aa4c-4e1a-b682-5580e9204727\") " pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:07.709652 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.709619    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcz9h\" (UniqueName: \"kubernetes.io/projected/c0b1d023-aa4c-4e1a-b682-5580e9204727-kube-api-access-hcz9h\") pod \"authorino-75d556bd77-hz82m\" (UID: \"c0b1d023-aa4c-4e1a-b682-5580e9204727\") " pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:07.719735 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.719704    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcz9h\" (UniqueName: \"kubernetes.io/projected/c0b1d023-aa4c-4e1a-b682-5580e9204727-kube-api-access-hcz9h\") pod \"authorino-75d556bd77-hz82m\" (UID: \"c0b1d023-aa4c-4e1a-b682-5580e9204727\") " pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:07.771295 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.771233    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z" event={"ID":"953c7e2e-5e2b-447d-a82a-dbe02d314571","Type":"ContainerStarted","Data":"6a3b2172257310b7fcc2f74abbb2dfdfa8d50ff743b4c3ab042b18b8a4534abc"}
Jul 29 01:06:07.841236 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:07.841203    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:08.020478 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:08.020448    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz82m"]
Jul 29 01:06:08.023248 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:08.023221    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0b1d023_aa4c_4e1a_b682_5580e9204727.slice/crio-47370fb7bc07582a622c5b7cece12b35ec2c67274727f3dc901168fb5cd0ba32 WatchSource:0}: Error finding container 47370fb7bc07582a622c5b7cece12b35ec2c67274727f3dc901168fb5cd0ba32: Status 404 returned error can't find the container with id 47370fb7bc07582a622c5b7cece12b35ec2c67274727f3dc901168fb5cd0ba32
Jul 29 01:06:08.781010 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:08.780933    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz82m" event={"ID":"c0b1d023-aa4c-4e1a-b682-5580e9204727","Type":"ContainerStarted","Data":"47370fb7bc07582a622c5b7cece12b35ec2c67274727f3dc901168fb5cd0ba32"}
Jul 29 01:06:12.808562 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:12.808523    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz82m" event={"ID":"c0b1d023-aa4c-4e1a-b682-5580e9204727","Type":"ContainerStarted","Data":"41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6"}
Jul 29 01:06:12.809982 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:12.809961    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z" event={"ID":"953c7e2e-5e2b-447d-a82a-dbe02d314571","Type":"ContainerStarted","Data":"f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0"}
Jul 29 01:06:12.810106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:12.810090    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:12.828256 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:12.828209    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hz82m" podStartSLOduration=1.856600642 podStartE2EDuration="5.828194744s" totalImagesPullingTime="3.971594102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:12.8263772 +0000 UTC m=+702.157026808" watchObservedRunningTime="2026-07-29 01:06:12.828194744 +0000 UTC m=+702.158844351"
Jul 29 01:06:12.846950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:12.846905    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z" podStartSLOduration=1.790311174 podStartE2EDuration="6.84688823s" totalImagesPullingTime="5.056577056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:12.844917991 +0000 UTC m=+702.175567596" watchObservedRunningTime="2026-07-29 01:06:12.84688823 +0000 UTC m=+702.177537835"
Jul 29 01:06:13.704596 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:13.704559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:23.472898 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:23.472814    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:23.473426 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:23.473041    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z" podUID="953c7e2e-5e2b-447d-a82a-dbe02d314571" containerName="limitador" containerID="cri-o://f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0" gracePeriod=30
Jul 29 01:06:23.474876 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:23.474852    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:24.021674 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.021648    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:24.026116 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.026092    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:24.168411 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.168318    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953c7e2e-5e2b-447d-a82a-dbe02d314571-config-file\") pod \"953c7e2e-5e2b-447d-a82a-dbe02d314571\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") "
Jul 29 01:06:24.168411 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.168375    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khrss\" (UniqueName: \"kubernetes.io/projected/953c7e2e-5e2b-447d-a82a-dbe02d314571-kube-api-access-khrss\") pod \"953c7e2e-5e2b-447d-a82a-dbe02d314571\" (UID: \"953c7e2e-5e2b-447d-a82a-dbe02d314571\") "
Jul 29 01:06:24.168709 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.168683    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953c7e2e-5e2b-447d-a82a-dbe02d314571-config-file" (OuterVolumeSpecName: "config-file") pod "953c7e2e-5e2b-447d-a82a-dbe02d314571" (UID: "953c7e2e-5e2b-447d-a82a-dbe02d314571"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:06:24.170375 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.170353    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953c7e2e-5e2b-447d-a82a-dbe02d314571-kube-api-access-khrss" (OuterVolumeSpecName: "kube-api-access-khrss") pod "953c7e2e-5e2b-447d-a82a-dbe02d314571" (UID: "953c7e2e-5e2b-447d-a82a-dbe02d314571"). InnerVolumeSpecName "kube-api-access-khrss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:06:24.269091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.269049    2555 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953c7e2e-5e2b-447d-a82a-dbe02d314571-config-file\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:06:24.269091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.269086    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khrss\" (UniqueName: \"kubernetes.io/projected/953c7e2e-5e2b-447d-a82a-dbe02d314571-kube-api-access-khrss\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:06:24.868631 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.868600    2555 generic.go:356] "Generic (PLEG): container finished" podID="953c7e2e-5e2b-447d-a82a-dbe02d314571" containerID="f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0" exitCode=0
Jul 29 01:06:24.869086 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.868646    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"
Jul 29 01:06:24.869086 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.868656    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z" event={"ID":"953c7e2e-5e2b-447d-a82a-dbe02d314571","Type":"ContainerDied","Data":"f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0"}
Jul 29 01:06:24.869086 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.868697    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hqw2z" event={"ID":"953c7e2e-5e2b-447d-a82a-dbe02d314571","Type":"ContainerDied","Data":"6a3b2172257310b7fcc2f74abbb2dfdfa8d50ff743b4c3ab042b18b8a4534abc"}
Jul 29 01:06:24.869086 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.868718    2555 scope.go:117] "RemoveContainer" containerID="f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0"
Jul 29 01:06:24.877230 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.877212    2555 scope.go:117] "RemoveContainer" containerID="f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0"
Jul 29 01:06:24.877491 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:06:24.877473    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0\": container with ID starting with f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0 not found: ID does not exist" containerID="f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0"
Jul 29 01:06:24.877556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.877500    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0"} err="failed to get container status \"f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0\": rpc error: code = NotFound desc = could not find container \"f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0\": container with ID starting with f6df54dec5e9466174f6ee158c4b0a5b1f84cb5e8337f8f7f8ed83abb88cc1c0 not found: ID does not exist"
Jul 29 01:06:24.895250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.895225    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:24.899069 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:24.899043    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hqw2z"]
Jul 29 01:06:25.189031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:25.189003    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953c7e2e-5e2b-447d-a82a-dbe02d314571" path="/var/lib/kubelet/pods/953c7e2e-5e2b-447d-a82a-dbe02d314571/volumes"
Jul 29 01:06:26.519528 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:26.519491    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:27.285674 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.285635    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:27.383050 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.383010    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:27.780449 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.780415    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"]
Jul 29 01:06:27.780976 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.780911    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953c7e2e-5e2b-447d-a82a-dbe02d314571" containerName="limitador"
Jul 29 01:06:27.780976 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.780931    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="953c7e2e-5e2b-447d-a82a-dbe02d314571" containerName="limitador"
Jul 29 01:06:27.781105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.781040    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="953c7e2e-5e2b-447d-a82a-dbe02d314571" containerName="limitador"
Jul 29 01:06:27.784306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.784261    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:27.787134 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.787117    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 01:06:27.787371 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.787353    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 01:06:27.788446 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.788432    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hflmp\""
Jul 29 01:06:27.788508 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.788474    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 01:06:27.795758 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.795738    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"]
Jul 29 01:06:27.808980 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.808960    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:27.903583 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.903550    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/40659dba-adc5-4d4f-bc5a-536e27bec534-data\") pod \"postgres-69fb4594cf-7lvmc\" (UID: \"40659dba-adc5-4d4f-bc5a-536e27bec534\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:27.903750 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:27.903589    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g48k8\" (UniqueName: \"kubernetes.io/projected/40659dba-adc5-4d4f-bc5a-536e27bec534-kube-api-access-g48k8\") pod \"postgres-69fb4594cf-7lvmc\" (UID: \"40659dba-adc5-4d4f-bc5a-536e27bec534\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:28.004414 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.004384    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/40659dba-adc5-4d4f-bc5a-536e27bec534-data\") pod \"postgres-69fb4594cf-7lvmc\" (UID: \"40659dba-adc5-4d4f-bc5a-536e27bec534\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:28.004535 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.004422    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g48k8\" (UniqueName: \"kubernetes.io/projected/40659dba-adc5-4d4f-bc5a-536e27bec534-kube-api-access-g48k8\") pod \"postgres-69fb4594cf-7lvmc\" (UID: \"40659dba-adc5-4d4f-bc5a-536e27bec534\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:28.004749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.004729    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/40659dba-adc5-4d4f-bc5a-536e27bec534-data\") pod \"postgres-69fb4594cf-7lvmc\" (UID: \"40659dba-adc5-4d4f-bc5a-536e27bec534\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:28.013271 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.013251    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g48k8\" (UniqueName: \"kubernetes.io/projected/40659dba-adc5-4d4f-bc5a-536e27bec534-kube-api-access-g48k8\") pod \"postgres-69fb4594cf-7lvmc\" (UID: \"40659dba-adc5-4d4f-bc5a-536e27bec534\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:28.094494 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.094393    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:28.226007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.225975    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"]
Jul 29 01:06:28.229122 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:28.229102    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40659dba_adc5_4d4f_bc5a_536e27bec534.slice/crio-631b5ffcef816447fd76fd9b5c2af10a4a545805c33fe93cc7753fda9baec77b WatchSource:0}: Error finding container 631b5ffcef816447fd76fd9b5c2af10a4a545805c33fe93cc7753fda9baec77b: Status 404 returned error can't find the container with id 631b5ffcef816447fd76fd9b5c2af10a4a545805c33fe93cc7753fda9baec77b
Jul 29 01:06:28.887818 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:28.887782    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc" event={"ID":"40659dba-adc5-4d4f-bc5a-536e27bec534","Type":"ContainerStarted","Data":"631b5ffcef816447fd76fd9b5c2af10a4a545805c33fe93cc7753fda9baec77b"}
Jul 29 01:06:33.591920 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:33.591887    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 01:06:33.916009 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:33.915908    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc" event={"ID":"40659dba-adc5-4d4f-bc5a-536e27bec534","Type":"ContainerStarted","Data":"57d262f8aeae499151f496d901981b07392b7a021cd05c36695f3c797b439440"}
Jul 29 01:06:33.916009 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:33.915972    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:33.932488 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:33.932434    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc" podStartSLOduration=1.574277215 podStartE2EDuration="6.932418878s" totalImagesPullingTime="5.358141663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:33.931234291 +0000 UTC m=+723.261883898" watchObservedRunningTime="2026-07-29 01:06:33.932418878 +0000 UTC m=+723.263068484"
Jul 29 01:06:39.949120 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:39.949091    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7lvmc"
Jul 29 01:06:40.271575 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.271488    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:40.736755 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.736723    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5494cb5d86-ffqxz"]
Jul 29 01:06:40.744843 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.744821    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:40.749720 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.749696    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5494cb5d86-ffqxz"]
Jul 29 01:06:40.756991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.756960    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:40.808594 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.808555    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncltf\" (UniqueName: \"kubernetes.io/projected/13c38ae9-5830-4a34-99b9-8b21c0cec2c2-kube-api-access-ncltf\") pod \"authorino-5494cb5d86-ffqxz\" (UID: \"13c38ae9-5830-4a34-99b9-8b21c0cec2c2\") " pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:40.909063 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.909020    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncltf\" (UniqueName: \"kubernetes.io/projected/13c38ae9-5830-4a34-99b9-8b21c0cec2c2-kube-api-access-ncltf\") pod \"authorino-5494cb5d86-ffqxz\" (UID: \"13c38ae9-5830-4a34-99b9-8b21c0cec2c2\") " pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:40.918941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:40.918904    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncltf\" (UniqueName: \"kubernetes.io/projected/13c38ae9-5830-4a34-99b9-8b21c0cec2c2-kube-api-access-ncltf\") pod \"authorino-5494cb5d86-ffqxz\" (UID: \"13c38ae9-5830-4a34-99b9-8b21c0cec2c2\") " pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:41.054587 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.054507    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:41.130205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.130171    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5494cb5d86-ffqxz"]
Jul 29 01:06:41.173074 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.173042    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bcdc844c4-j4njv"]
Jul 29 01:06:41.177189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.177167    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.180904 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.180886    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 01:06:41.192120 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.192098    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5494cb5d86-ffqxz"]
Jul 29 01:06:41.192224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.192138    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bcdc844c4-j4njv"]
Jul 29 01:06:41.193187 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:41.193160    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13c38ae9_5830_4a34_99b9_8b21c0cec2c2.slice/crio-2ef487317eee737ee6d46e3c31243c9b63aa8616bd790ed1910ca9926d203924 WatchSource:0}: Error finding container 2ef487317eee737ee6d46e3c31243c9b63aa8616bd790ed1910ca9926d203924: Status 404 returned error can't find the container with id 2ef487317eee737ee6d46e3c31243c9b63aa8616bd790ed1910ca9926d203924
Jul 29 01:06:41.211011 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.210981    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07030d7b-cf5d-41f8-98cd-5af28e1413f0-tls-cert\") pod \"authorino-bcdc844c4-j4njv\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") " pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.211113 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.211048    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss5s7\" (UniqueName: \"kubernetes.io/projected/07030d7b-cf5d-41f8-98cd-5af28e1413f0-kube-api-access-ss5s7\") pod \"authorino-bcdc844c4-j4njv\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") " pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.311991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.311904    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07030d7b-cf5d-41f8-98cd-5af28e1413f0-tls-cert\") pod \"authorino-bcdc844c4-j4njv\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") " pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.312131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.312004    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss5s7\" (UniqueName: \"kubernetes.io/projected/07030d7b-cf5d-41f8-98cd-5af28e1413f0-kube-api-access-ss5s7\") pod \"authorino-bcdc844c4-j4njv\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") " pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.314335 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.314307    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07030d7b-cf5d-41f8-98cd-5af28e1413f0-tls-cert\") pod \"authorino-bcdc844c4-j4njv\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") " pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.322026 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.321986    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss5s7\" (UniqueName: \"kubernetes.io/projected/07030d7b-cf5d-41f8-98cd-5af28e1413f0-kube-api-access-ss5s7\") pod \"authorino-bcdc844c4-j4njv\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") " pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.487177 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.487141    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:06:41.637381 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.637322    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bcdc844c4-j4njv"]
Jul 29 01:06:41.639977 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:41.639948    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07030d7b_cf5d_41f8_98cd_5af28e1413f0.slice/crio-40c96d2566c5bad09fce4eb34a802dda2faba508d8887050997a5c8b6c042be8 WatchSource:0}: Error finding container 40c96d2566c5bad09fce4eb34a802dda2faba508d8887050997a5c8b6c042be8: Status 404 returned error can't find the container with id 40c96d2566c5bad09fce4eb34a802dda2faba508d8887050997a5c8b6c042be8
Jul 29 01:06:41.953153 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.953118    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcdc844c4-j4njv" event={"ID":"07030d7b-cf5d-41f8-98cd-5af28e1413f0","Type":"ContainerStarted","Data":"40c96d2566c5bad09fce4eb34a802dda2faba508d8887050997a5c8b6c042be8"}
Jul 29 01:06:41.954609 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.954571    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5494cb5d86-ffqxz" event={"ID":"13c38ae9-5830-4a34-99b9-8b21c0cec2c2","Type":"ContainerStarted","Data":"6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06"}
Jul 29 01:06:41.954739 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.954617    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5494cb5d86-ffqxz" event={"ID":"13c38ae9-5830-4a34-99b9-8b21c0cec2c2","Type":"ContainerStarted","Data":"2ef487317eee737ee6d46e3c31243c9b63aa8616bd790ed1910ca9926d203924"}
Jul 29 01:06:41.954739 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.954648    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5494cb5d86-ffqxz" podUID="13c38ae9-5830-4a34-99b9-8b21c0cec2c2" containerName="authorino" containerID="cri-o://6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06" gracePeriod=30
Jul 29 01:06:41.977827 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:41.977777    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5494cb5d86-ffqxz" podStartSLOduration=1.667755162 podStartE2EDuration="1.977761349s" totalImagesPullingTime="310.006187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:41.975933391 +0000 UTC m=+731.306582998" watchObservedRunningTime="2026-07-29 01:06:41.977761349 +0000 UTC m=+731.308410955"
Jul 29 01:06:42.209480 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.209413    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:42.219866 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.219842    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncltf\" (UniqueName: \"kubernetes.io/projected/13c38ae9-5830-4a34-99b9-8b21c0cec2c2-kube-api-access-ncltf\") pod \"13c38ae9-5830-4a34-99b9-8b21c0cec2c2\" (UID: \"13c38ae9-5830-4a34-99b9-8b21c0cec2c2\") "
Jul 29 01:06:42.221825 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.221792    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c38ae9-5830-4a34-99b9-8b21c0cec2c2-kube-api-access-ncltf" (OuterVolumeSpecName: "kube-api-access-ncltf") pod "13c38ae9-5830-4a34-99b9-8b21c0cec2c2" (UID: "13c38ae9-5830-4a34-99b9-8b21c0cec2c2"). InnerVolumeSpecName "kube-api-access-ncltf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:06:42.320868 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.320822    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncltf\" (UniqueName: \"kubernetes.io/projected/13c38ae9-5830-4a34-99b9-8b21c0cec2c2-kube-api-access-ncltf\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:06:42.960478 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.960438    2555 generic.go:356] "Generic (PLEG): container finished" podID="13c38ae9-5830-4a34-99b9-8b21c0cec2c2" containerID="6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06" exitCode=0
Jul 29 01:06:42.960655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.960494    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5494cb5d86-ffqxz"
Jul 29 01:06:42.960655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.960524    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5494cb5d86-ffqxz" event={"ID":"13c38ae9-5830-4a34-99b9-8b21c0cec2c2","Type":"ContainerDied","Data":"6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06"}
Jul 29 01:06:42.960655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.960568    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5494cb5d86-ffqxz" event={"ID":"13c38ae9-5830-4a34-99b9-8b21c0cec2c2","Type":"ContainerDied","Data":"2ef487317eee737ee6d46e3c31243c9b63aa8616bd790ed1910ca9926d203924"}
Jul 29 01:06:42.960655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.960587    2555 scope.go:117] "RemoveContainer" containerID="6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06"
Jul 29 01:06:42.962124 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.962094    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcdc844c4-j4njv" event={"ID":"07030d7b-cf5d-41f8-98cd-5af28e1413f0","Type":"ContainerStarted","Data":"00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f"}
Jul 29 01:06:42.969021 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.969005    2555 scope.go:117] "RemoveContainer" containerID="6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06"
Jul 29 01:06:42.969271 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:06:42.969253    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06\": container with ID starting with 6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06 not found: ID does not exist" containerID="6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06"
Jul 29 01:06:42.969355 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.969300    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06"} err="failed to get container status \"6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06\": rpc error: code = NotFound desc = could not find container \"6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06\": container with ID starting with 6027cff21e703d6d31a14caa20da943cd083615c39f75df5b6092be455c56f06 not found: ID does not exist"
Jul 29 01:06:42.980942 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.980900    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-bcdc844c4-j4njv" podStartSLOduration=1.627526359 podStartE2EDuration="1.980887299s" totalImagesPullingTime="353.36094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:42.979513473 +0000 UTC m=+732.310163081" watchObservedRunningTime="2026-07-29 01:06:42.980887299 +0000 UTC m=+732.311536905"
Jul 29 01:06:42.994673 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:42.994647    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5494cb5d86-ffqxz"]
Jul 29 01:06:43.002685 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.002658    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5494cb5d86-ffqxz"]
Jul 29 01:06:43.009233 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.009213    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz82m"]
Jul 29 01:06:43.009443 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.009420    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hz82m" podUID="c0b1d023-aa4c-4e1a-b682-5580e9204727" containerName="authorino" containerID="cri-o://41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6" gracePeriod=30
Jul 29 01:06:43.189914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.189818    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13c38ae9-5830-4a34-99b9-8b21c0cec2c2" path="/var/lib/kubelet/pods/13c38ae9-5830-4a34-99b9-8b21c0cec2c2/volumes"
Jul 29 01:06:43.254977 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.254949    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:43.327831 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.327799    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcz9h\" (UniqueName: \"kubernetes.io/projected/c0b1d023-aa4c-4e1a-b682-5580e9204727-kube-api-access-hcz9h\") pod \"c0b1d023-aa4c-4e1a-b682-5580e9204727\" (UID: \"c0b1d023-aa4c-4e1a-b682-5580e9204727\") "
Jul 29 01:06:43.329701 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.329678    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b1d023-aa4c-4e1a-b682-5580e9204727-kube-api-access-hcz9h" (OuterVolumeSpecName: "kube-api-access-hcz9h") pod "c0b1d023-aa4c-4e1a-b682-5580e9204727" (UID: "c0b1d023-aa4c-4e1a-b682-5580e9204727"). InnerVolumeSpecName "kube-api-access-hcz9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:06:43.429180 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.429138    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcz9h\" (UniqueName: \"kubernetes.io/projected/c0b1d023-aa4c-4e1a-b682-5580e9204727-kube-api-access-hcz9h\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:06:43.968155 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.968116    2555 generic.go:356] "Generic (PLEG): container finished" podID="c0b1d023-aa4c-4e1a-b682-5580e9204727" containerID="41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6" exitCode=0
Jul 29 01:06:43.968155 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.968153    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hz82m"
Jul 29 01:06:43.968403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.968203    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz82m" event={"ID":"c0b1d023-aa4c-4e1a-b682-5580e9204727","Type":"ContainerDied","Data":"41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6"}
Jul 29 01:06:43.968403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.968243    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hz82m" event={"ID":"c0b1d023-aa4c-4e1a-b682-5580e9204727","Type":"ContainerDied","Data":"47370fb7bc07582a622c5b7cece12b35ec2c67274727f3dc901168fb5cd0ba32"}
Jul 29 01:06:43.968403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.968271    2555 scope.go:117] "RemoveContainer" containerID="41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6"
Jul 29 01:06:43.976601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.976586    2555 scope.go:117] "RemoveContainer" containerID="41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6"
Jul 29 01:06:43.976836 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:06:43.976819    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6\": container with ID starting with 41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6 not found: ID does not exist" containerID="41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6"
Jul 29 01:06:43.976888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.976844    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6"} err="failed to get container status \"41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6\": rpc error: code = NotFound desc = could not find container \"41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6\": container with ID starting with 41d6d6ba7ad30e05250a100dff03566b8c15e42dc7cd13f439d433947d697de6 not found: ID does not exist"
Jul 29 01:06:43.989111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.989090    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz82m"]
Jul 29 01:06:43.992547 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:43.992524    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hz82m"]
Jul 29 01:06:45.189583 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:45.189546    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0b1d023-aa4c-4e1a-b682-5580e9204727" path="/var/lib/kubelet/pods/c0b1d023-aa4c-4e1a-b682-5580e9204727/volumes"
Jul 29 01:06:47.528197 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.528160    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:47.530979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.530951    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-755f898f66-4kcpj"]
Jul 29 01:06:47.531342 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.531329    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c38ae9-5830-4a34-99b9-8b21c0cec2c2" containerName="authorino"
Jul 29 01:06:47.531402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.531344    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c38ae9-5830-4a34-99b9-8b21c0cec2c2" containerName="authorino"
Jul 29 01:06:47.531402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.531354    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b1d023-aa4c-4e1a-b682-5580e9204727" containerName="authorino"
Jul 29 01:06:47.531402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.531359    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b1d023-aa4c-4e1a-b682-5580e9204727" containerName="authorino"
Jul 29 01:06:47.531507 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.531421    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b1d023-aa4c-4e1a-b682-5580e9204727" containerName="authorino"
Jul 29 01:06:47.531507 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.531431    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c38ae9-5830-4a34-99b9-8b21c0cec2c2" containerName="authorino"
Jul 29 01:06:47.534811 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.534777    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.540991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.540960    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 01:06:47.541164 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.541061    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2w5vs\""
Jul 29 01:06:47.541244 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.541226    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 01:06:47.548832 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.548801    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-755f898f66-4kcpj"]
Jul 29 01:06:47.554644 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.554616    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9m7p\" (UniqueName: \"kubernetes.io/projected/a6031a76-4b81-45df-95c9-750ad9e27583-kube-api-access-z9m7p\") pod \"maas-controller-755f898f66-4kcpj\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") " pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.554774 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.554671    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6031a76-4b81-45df-95c9-750ad9e27583-webhook-cert\") pod \"maas-controller-755f898f66-4kcpj\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") " pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.616033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.615992    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:47.655553 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.655522    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9m7p\" (UniqueName: \"kubernetes.io/projected/a6031a76-4b81-45df-95c9-750ad9e27583-kube-api-access-z9m7p\") pod \"maas-controller-755f898f66-4kcpj\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") " pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.655716 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.655587    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6031a76-4b81-45df-95c9-750ad9e27583-webhook-cert\") pod \"maas-controller-755f898f66-4kcpj\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") " pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.657999 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.657963    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6031a76-4b81-45df-95c9-750ad9e27583-webhook-cert\") pod \"maas-controller-755f898f66-4kcpj\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") " pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.666849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.666826    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9m7p\" (UniqueName: \"kubernetes.io/projected/a6031a76-4b81-45df-95c9-750ad9e27583-kube-api-access-z9m7p\") pod \"maas-controller-755f898f66-4kcpj\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") " pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.697514 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.697472    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:47.702415 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.702385    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d584db87-bmsh4"]
Jul 29 01:06:47.704966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.704945    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.716741 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.716705    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d584db87-bmsh4"]
Jul 29 01:06:47.756040 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.756003    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfc19f8-af3f-4575-a586-7aa2aa672130-webhook-cert\") pod \"maas-controller-6d584db87-bmsh4\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") " pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.756230 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.756060    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gdff\" (UniqueName: \"kubernetes.io/projected/5dfc19f8-af3f-4575-a586-7aa2aa672130-kube-api-access-5gdff\") pod \"maas-controller-6d584db87-bmsh4\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") " pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.756451 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.756430    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:47.849461 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.849363    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:47.857445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.857414    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfc19f8-af3f-4575-a586-7aa2aa672130-webhook-cert\") pod \"maas-controller-6d584db87-bmsh4\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") " pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.857578 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.857460    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gdff\" (UniqueName: \"kubernetes.io/projected/5dfc19f8-af3f-4575-a586-7aa2aa672130-kube-api-access-5gdff\") pod \"maas-controller-6d584db87-bmsh4\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") " pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.859811 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.859788    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfc19f8-af3f-4575-a586-7aa2aa672130-webhook-cert\") pod \"maas-controller-6d584db87-bmsh4\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") " pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.865658 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.865632    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gdff\" (UniqueName: \"kubernetes.io/projected/5dfc19f8-af3f-4575-a586-7aa2aa672130-kube-api-access-5gdff\") pod \"maas-controller-6d584db87-bmsh4\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") " pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:47.980321 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.980286    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-755f898f66-4kcpj"]
Jul 29 01:06:47.983625 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:47.983599    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6031a76_4b81_45df_95c9_750ad9e27583.slice/crio-539a9e49e3fe9cda4869ccadd92d686f08aeb5e2e0af5004e4001fcfda438bc6 WatchSource:0}: Error finding container 539a9e49e3fe9cda4869ccadd92d686f08aeb5e2e0af5004e4001fcfda438bc6: Status 404 returned error can't find the container with id 539a9e49e3fe9cda4869ccadd92d686f08aeb5e2e0af5004e4001fcfda438bc6
Jul 29 01:06:47.991446 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:47.991420    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-755f898f66-4kcpj" event={"ID":"a6031a76-4b81-45df-95c9-750ad9e27583","Type":"ContainerStarted","Data":"539a9e49e3fe9cda4869ccadd92d686f08aeb5e2e0af5004e4001fcfda438bc6"}
Jul 29 01:06:48.016269 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:48.016238    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:48.150342 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:48.150247    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d584db87-bmsh4"]
Jul 29 01:06:48.153256 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:06:48.153229    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfc19f8_af3f_4575_a586_7aa2aa672130.slice/crio-aef528f32b2cb498b0564fa8bc33c8548949f24d83336b257b17632ddb268244 WatchSource:0}: Error finding container aef528f32b2cb498b0564fa8bc33c8548949f24d83336b257b17632ddb268244: Status 404 returned error can't find the container with id aef528f32b2cb498b0564fa8bc33c8548949f24d83336b257b17632ddb268244
Jul 29 01:06:49.009735 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:49.009692    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-bmsh4" event={"ID":"5dfc19f8-af3f-4575-a586-7aa2aa672130","Type":"ContainerStarted","Data":"aef528f32b2cb498b0564fa8bc33c8548949f24d83336b257b17632ddb268244"}
Jul 29 01:06:50.702144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:50.702106    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:50.799452 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:50.799366    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:06:51.023047 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:51.023009    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-755f898f66-4kcpj" event={"ID":"a6031a76-4b81-45df-95c9-750ad9e27583","Type":"ContainerStarted","Data":"fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f"}
Jul 29 01:06:51.023228 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:51.023212    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:06:51.024436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:51.024413    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-bmsh4" event={"ID":"5dfc19f8-af3f-4575-a586-7aa2aa672130","Type":"ContainerStarted","Data":"c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79"}
Jul 29 01:06:51.024576 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:51.024537    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:06:51.041808 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:51.041762    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-755f898f66-4kcpj" podStartSLOduration=1.508734005 podStartE2EDuration="4.041745824s" totalImagesPullingTime="2.533011819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:51.040145227 +0000 UTC m=+740.370794833" watchObservedRunningTime="2026-07-29 01:06:51.041745824 +0000 UTC m=+740.372395431"
Jul 29 01:06:51.059438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:06:51.059354    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d584db87-bmsh4" podStartSLOduration=1.6900579329999998 podStartE2EDuration="4.0593423s" totalImagesPullingTime="2.369284367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:06:51.057708938 +0000 UTC m=+740.388358545" watchObservedRunningTime="2026-07-29 01:06:51.0593423 +0000 UTC m=+740.389991906"
Jul 29 01:07:02.032344 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.032307    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:07:02.032816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.032391    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:07:02.092732 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.092690    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-755f898f66-4kcpj"]
Jul 29 01:07:02.092970 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.092920    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-755f898f66-4kcpj" podUID="a6031a76-4b81-45df-95c9-750ad9e27583" containerName="manager" containerID="cri-o://fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f" gracePeriod=30
Jul 29 01:07:02.351966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.351936    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:07:02.373433 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.373401    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9m7p\" (UniqueName: \"kubernetes.io/projected/a6031a76-4b81-45df-95c9-750ad9e27583-kube-api-access-z9m7p\") pod \"a6031a76-4b81-45df-95c9-750ad9e27583\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") "
Jul 29 01:07:02.373601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.373445    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6031a76-4b81-45df-95c9-750ad9e27583-webhook-cert\") pod \"a6031a76-4b81-45df-95c9-750ad9e27583\" (UID: \"a6031a76-4b81-45df-95c9-750ad9e27583\") "
Jul 29 01:07:02.375654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.375622    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6031a76-4b81-45df-95c9-750ad9e27583-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a6031a76-4b81-45df-95c9-750ad9e27583" (UID: "a6031a76-4b81-45df-95c9-750ad9e27583"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:07:02.375763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.375665    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6031a76-4b81-45df-95c9-750ad9e27583-kube-api-access-z9m7p" (OuterVolumeSpecName: "kube-api-access-z9m7p") pod "a6031a76-4b81-45df-95c9-750ad9e27583" (UID: "a6031a76-4b81-45df-95c9-750ad9e27583"). InnerVolumeSpecName "kube-api-access-z9m7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:07:02.475253 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.475208    2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a6031a76-4b81-45df-95c9-750ad9e27583-webhook-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:07:02.475253 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:02.475246    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9m7p\" (UniqueName: \"kubernetes.io/projected/a6031a76-4b81-45df-95c9-750ad9e27583-kube-api-access-z9m7p\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:07:03.092599 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.092565    2555 generic.go:356] "Generic (PLEG): container finished" podID="a6031a76-4b81-45df-95c9-750ad9e27583" containerID="fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f" exitCode=0
Jul 29 01:07:03.093091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.092609    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-755f898f66-4kcpj"
Jul 29 01:07:03.093091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.092652    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-755f898f66-4kcpj" event={"ID":"a6031a76-4b81-45df-95c9-750ad9e27583","Type":"ContainerDied","Data":"fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f"}
Jul 29 01:07:03.093091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.092713    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-755f898f66-4kcpj" event={"ID":"a6031a76-4b81-45df-95c9-750ad9e27583","Type":"ContainerDied","Data":"539a9e49e3fe9cda4869ccadd92d686f08aeb5e2e0af5004e4001fcfda438bc6"}
Jul 29 01:07:03.093091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.092747    2555 scope.go:117] "RemoveContainer" containerID="fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f"
Jul 29 01:07:03.101645 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.101628    2555 scope.go:117] "RemoveContainer" containerID="fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f"
Jul 29 01:07:03.101886 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:07:03.101868    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f\": container with ID starting with fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f not found: ID does not exist" containerID="fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f"
Jul 29 01:07:03.101946 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.101898    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f"} err="failed to get container status \"fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f\": rpc error: code = NotFound desc = could not find container \"fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f\": container with ID starting with fffca11e5d3fad63547bca095a80a0f3e311560dd1396ffc76f0ee8f7528f05f not found: ID does not exist"
Jul 29 01:07:03.113539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.113512    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-755f898f66-4kcpj"]
Jul 29 01:07:03.116861 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.116838    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-755f898f66-4kcpj"]
Jul 29 01:07:03.188690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:03.188658    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6031a76-4b81-45df-95c9-750ad9e27583" path="/var/lib/kubelet/pods/a6031a76-4b81-45df-95c9-750ad9e27583/volumes"
Jul 29 01:07:17.474339 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:17.474306    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:23.345618 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:23.345577    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:23.444793 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:23.444759    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:24.024171 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.024143    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:24.140426 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.140395    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"]
Jul 29 01:07:24.140749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.140737    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6031a76-4b81-45df-95c9-750ad9e27583" containerName="manager"
Jul 29 01:07:24.140794 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.140751    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6031a76-4b81-45df-95c9-750ad9e27583" containerName="manager"
Jul 29 01:07:24.140828 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.140817    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6031a76-4b81-45df-95c9-750ad9e27583" containerName="manager"
Jul 29 01:07:24.168517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.168467    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"]
Jul 29 01:07:24.168657 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.168598    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.173391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.173367    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 01:07:24.173531 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.173437    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-drtj4\""
Jul 29 01:07:24.251250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.251227    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f09ae-2e69-4896-8148-0000a8858fbb-openshift-service-ca\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.251434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.251263    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt7d5\" (UniqueName: \"kubernetes.io/projected/470f09ae-2e69-4896-8148-0000a8858fbb-kube-api-access-nt7d5\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.251434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.251387    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f09ae-2e69-4896-8148-0000a8858fbb-maas-api-tls\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.352705 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.352638    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f09ae-2e69-4896-8148-0000a8858fbb-openshift-service-ca\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.352705 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.352679    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt7d5\" (UniqueName: \"kubernetes.io/projected/470f09ae-2e69-4896-8148-0000a8858fbb-kube-api-access-nt7d5\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.353068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.352730    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f09ae-2e69-4896-8148-0000a8858fbb-maas-api-tls\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.353417 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.353398    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/470f09ae-2e69-4896-8148-0000a8858fbb-openshift-service-ca\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.355013 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.354997    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/470f09ae-2e69-4896-8148-0000a8858fbb-maas-api-tls\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.360362 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.360344    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt7d5\" (UniqueName: \"kubernetes.io/projected/470f09ae-2e69-4896-8148-0000a8858fbb-kube-api-access-nt7d5\") pod \"maas-api-dfd9875cc-7rgsp\" (UID: \"470f09ae-2e69-4896-8148-0000a8858fbb\") " pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.455945 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.455917    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:24.479928 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.479899    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:24.610966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.610880    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:24.617630 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.617603    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"]
Jul 29 01:07:24.620093 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:07:24.620066    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470f09ae_2e69_4896_8148_0000a8858fbb.slice/crio-dc287850529eff409afbde31d9c53be6cf8f6e62680e4be28d95b944679ae8c4 WatchSource:0}: Error finding container dc287850529eff409afbde31d9c53be6cf8f6e62680e4be28d95b944679ae8c4: Status 404 returned error can't find the container with id dc287850529eff409afbde31d9c53be6cf8f6e62680e4be28d95b944679ae8c4
Jul 29 01:07:24.706649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.706619    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:24.710681 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.710657    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"]
Jul 29 01:07:24.715367 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.715350    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:24.717851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.717831    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 01:07:24.722508 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.722488    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"]
Jul 29 01:07:24.751718 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.751688    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-z8k9t"]
Jul 29 01:07:24.756154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.756131    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.759223 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.759202    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-22b9r\""
Jul 29 01:07:24.764748 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.764727    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-z8k9t"]
Jul 29 01:07:24.856916 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.856882    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nckj\" (UniqueName: \"kubernetes.io/projected/83855546-55fb-4f87-948b-086b183adc64-kube-api-access-9nckj\") pod \"payload-processing-5cc6cd444b-z8k9t\" (UID: \"83855546-55fb-4f87-948b-086b183adc64\") " pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.857130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.857038    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83855546-55fb-4f87-948b-086b183adc64-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-z8k9t\" (UID: \"83855546-55fb-4f87-948b-086b183adc64\") " pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.857130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.857083    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hfch\" (UniqueName: \"kubernetes.io/projected/d418b254-d302-4d09-8c9b-8abc24d6ec18-kube-api-access-8hfch\") pod \"payload-pre-processing-687dbb6cf6-8gzhg\" (UID: \"d418b254-d302-4d09-8c9b-8abc24d6ec18\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:24.857130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.857125    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d418b254-d302-4d09-8c9b-8abc24d6ec18-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-8gzhg\" (UID: \"d418b254-d302-4d09-8c9b-8abc24d6ec18\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:24.957586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.957561    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hfch\" (UniqueName: \"kubernetes.io/projected/d418b254-d302-4d09-8c9b-8abc24d6ec18-kube-api-access-8hfch\") pod \"payload-pre-processing-687dbb6cf6-8gzhg\" (UID: \"d418b254-d302-4d09-8c9b-8abc24d6ec18\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:24.957746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.957593    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d418b254-d302-4d09-8c9b-8abc24d6ec18-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-8gzhg\" (UID: \"d418b254-d302-4d09-8c9b-8abc24d6ec18\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:24.957746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.957640    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nckj\" (UniqueName: \"kubernetes.io/projected/83855546-55fb-4f87-948b-086b183adc64-kube-api-access-9nckj\") pod \"payload-processing-5cc6cd444b-z8k9t\" (UID: \"83855546-55fb-4f87-948b-086b183adc64\") " pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.957746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.957728    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83855546-55fb-4f87-948b-086b183adc64-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-z8k9t\" (UID: \"83855546-55fb-4f87-948b-086b183adc64\") " pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.958561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.958539    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d418b254-d302-4d09-8c9b-8abc24d6ec18-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-8gzhg\" (UID: \"d418b254-d302-4d09-8c9b-8abc24d6ec18\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:24.958607 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.958539    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83855546-55fb-4f87-948b-086b183adc64-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-z8k9t\" (UID: \"83855546-55fb-4f87-948b-086b183adc64\") " pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.967872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.967850    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nckj\" (UniqueName: \"kubernetes.io/projected/83855546-55fb-4f87-948b-086b183adc64-kube-api-access-9nckj\") pod \"payload-processing-5cc6cd444b-z8k9t\" (UID: \"83855546-55fb-4f87-948b-086b183adc64\") " pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:24.967872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:24.967859    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hfch\" (UniqueName: \"kubernetes.io/projected/d418b254-d302-4d09-8c9b-8abc24d6ec18-kube-api-access-8hfch\") pod \"payload-pre-processing-687dbb6cf6-8gzhg\" (UID: \"d418b254-d302-4d09-8c9b-8abc24d6ec18\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:25.028194 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:25.028167    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:25.068962 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:25.068882    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:25.173745 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:25.173714    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"]
Jul 29 01:07:25.176209 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:07:25.176177    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd418b254_d302_4d09_8c9b_8abc24d6ec18.slice/crio-615d4bb8b1ebd0041932f10a22a0d307f73e804e88bd293f40224e00f04befe2 WatchSource:0}: Error finding container 615d4bb8b1ebd0041932f10a22a0d307f73e804e88bd293f40224e00f04befe2: Status 404 returned error can't find the container with id 615d4bb8b1ebd0041932f10a22a0d307f73e804e88bd293f40224e00f04befe2
Jul 29 01:07:25.204101 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:25.203819    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg" event={"ID":"d418b254-d302-4d09-8c9b-8abc24d6ec18","Type":"ContainerStarted","Data":"615d4bb8b1ebd0041932f10a22a0d307f73e804e88bd293f40224e00f04befe2"}
Jul 29 01:07:25.205744 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:25.205710    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp" event={"ID":"470f09ae-2e69-4896-8148-0000a8858fbb","Type":"ContainerStarted","Data":"dc287850529eff409afbde31d9c53be6cf8f6e62680e4be28d95b944679ae8c4"}
Jul 29 01:07:25.221744 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:25.221561    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-z8k9t"]
Jul 29 01:07:25.224919 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:07:25.224887    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83855546_55fb_4f87_948b_086b183adc64.slice/crio-d769cbc1201fa1b2ad4420235e4d39991abfbde43b0659d863f757e2acf3c85d WatchSource:0}: Error finding container d769cbc1201fa1b2ad4420235e4d39991abfbde43b0659d863f757e2acf3c85d: Status 404 returned error can't find the container with id d769cbc1201fa1b2ad4420235e4d39991abfbde43b0659d863f757e2acf3c85d
Jul 29 01:07:26.215370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:26.215189    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t" event={"ID":"83855546-55fb-4f87-948b-086b183adc64","Type":"ContainerStarted","Data":"d769cbc1201fa1b2ad4420235e4d39991abfbde43b0659d863f757e2acf3c85d"}
Jul 29 01:07:27.221323 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:27.221265    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp" event={"ID":"470f09ae-2e69-4896-8148-0000a8858fbb","Type":"ContainerStarted","Data":"87b3dfec83a810280ca1855edfc682f648278d7d4b418177a06d6976f744fbca"}
Jul 29 01:07:27.221755 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:27.221464    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:27.239111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:27.239058    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp" podStartSLOduration=1.534859824 podStartE2EDuration="3.239037798s" totalImagesPullingTime="1.704177974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:07:27.237059007 +0000 UTC m=+776.567708611" watchObservedRunningTime="2026-07-29 01:07:27.239037798 +0000 UTC m=+776.569687409"
Jul 29 01:07:28.227790 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:28.227750    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg" event={"ID":"d418b254-d302-4d09-8c9b-8abc24d6ec18","Type":"ContainerStarted","Data":"731701c6b97f2c75d6e046a4460cd5bef1c803b2f34db493cd5161ec62db7323"}
Jul 29 01:07:28.228214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:28.227814    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:07:28.229271 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:28.229244    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t" event={"ID":"83855546-55fb-4f87-948b-086b183adc64","Type":"ContainerStarted","Data":"8d27062c41dc8dda7660bf1743bdaad0ac61c1be48421b3fe42b99f480d60d7e"}
Jul 29 01:07:28.229400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:28.229354    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:28.244712 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:28.244669    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg" podStartSLOduration=1.722731852 podStartE2EDuration="4.244650175s" totalImagesPullingTime="2.521918323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:07:28.242448523 +0000 UTC m=+777.573098130" watchObservedRunningTime="2026-07-29 01:07:28.244650175 +0000 UTC m=+777.575299785"
Jul 29 01:07:28.258106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:28.258066    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t" podStartSLOduration=1.7912196740000002 podStartE2EDuration="4.258053546s" totalImagesPullingTime="2.466833872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:07:28.257696278 +0000 UTC m=+777.588345888" watchObservedRunningTime="2026-07-29 01:07:28.258053546 +0000 UTC m=+777.588703152"
Jul 29 01:07:33.235409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:33.235376    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-dfd9875cc-7rgsp"
Jul 29 01:07:34.177216 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:34.177169    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:34.276604 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:34.276572    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:07:39.235877 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:39.235844    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-z8k9t"
Jul 29 01:07:39.236357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:07:39.235943    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-8gzhg"
Jul 29 01:08:00.155752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:00.155663    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:00.675743 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:00.675706    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:00.776620 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:00.776584    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:10.580413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:10.580377    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:26.787958 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.787915    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"]
Jul 29 01:08:26.791683 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.791660    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.795473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.795434    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 01:08:26.795473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.795435    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kjskh\""
Jul 29 01:08:26.795654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.795476    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 01:08:26.795654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.795506    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 01:08:26.800887 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.800862    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"]
Jul 29 01:08:26.820484 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.820453    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.820656 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.820503    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.820656 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.820535    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.820656 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.820585    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.820656 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.820605    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt8dj\" (UniqueName: \"kubernetes.io/projected/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-kube-api-access-nt8dj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.820871 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.820657    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.830077 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.830048    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:26.921372 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921328    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921567 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921399    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921567 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921442    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921567 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921467    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt8dj\" (UniqueName: \"kubernetes.io/projected/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-kube-api-access-nt8dj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921567 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921506    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921794 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921592    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921854 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921808    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921908 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921888    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.921975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.921954    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.923736 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.923707    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.923901 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.923884    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:26.929214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:26.929193    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt8dj\" (UniqueName: \"kubernetes.io/projected/fa6ab3e2-9490-4053-8d9b-4d0509b3b786-kube-api-access-nt8dj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p44wj\" (UID: \"fa6ab3e2-9490-4053-8d9b-4d0509b3b786\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:27.103039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:27.102948    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:27.237404 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:27.237378    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"]
Jul 29 01:08:27.240435 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:08:27.240404    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6ab3e2_9490_4053_8d9b_4d0509b3b786.slice/crio-fdde0abc3ac44db10addde878dbe56262d8adf33c50f229eabf057adb245222b WatchSource:0}: Error finding container fdde0abc3ac44db10addde878dbe56262d8adf33c50f229eabf057adb245222b: Status 404 returned error can't find the container with id fdde0abc3ac44db10addde878dbe56262d8adf33c50f229eabf057adb245222b
Jul 29 01:08:27.516002 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:27.515967    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj" event={"ID":"fa6ab3e2-9490-4053-8d9b-4d0509b3b786","Type":"ContainerStarted","Data":"fdde0abc3ac44db10addde878dbe56262d8adf33c50f229eabf057adb245222b"}
Jul 29 01:08:28.522377 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:28.522255    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj" event={"ID":"fa6ab3e2-9490-4053-8d9b-4d0509b3b786","Type":"ContainerStarted","Data":"8b13eac52da5dbe4dfce46b4c081a73681b99e78cb9f121899b5c5d0e43fae77"}
Jul 29 01:08:28.560318 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:28.560242    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj" podStartSLOduration=1.552941006 podStartE2EDuration="2.560222529s" totalImagesPullingTime="1.007281523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:08:28.558419722 +0000 UTC m=+837.889069328" watchObservedRunningTime="2026-07-29 01:08:28.560222529 +0000 UTC m=+837.890872134"
Jul 29 01:08:31.918539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:31.918501    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:37.103186 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:37.103149    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:37.103661 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:37.103479    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:37.115480 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:37.115451    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:37.577170 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:37.577131    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p44wj"
Jul 29 01:08:57.088320 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.088269    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"]
Jul 29 01:08:57.092772 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.092746    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.095569 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.095542    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 01:08:57.101133 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.101107    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"]
Jul 29 01:08:57.127826 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.127795    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:08:57.194806 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.194772    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.194974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.194832    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.194974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.194855    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwt6r\" (UniqueName: \"kubernetes.io/projected/73ec613f-7f12-460f-8584-9c39f969a8e1-kube-api-access-lwt6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.194974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.194888    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ec613f-7f12-460f-8584-9c39f969a8e1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.194974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.194941    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.195112 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.194980    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296134 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296093    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296134 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296139    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwt6r\" (UniqueName: \"kubernetes.io/projected/73ec613f-7f12-460f-8584-9c39f969a8e1-kube-api-access-lwt6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296172    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ec613f-7f12-460f-8584-9c39f969a8e1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296192    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296211    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296260    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296603 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296551    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296662 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296598    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.296662 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.296638    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.298499 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.298476    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73ec613f-7f12-460f-8584-9c39f969a8e1-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.298773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.298755    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73ec613f-7f12-460f-8584-9c39f969a8e1-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.303658 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.303636    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwt6r\" (UniqueName: \"kubernetes.io/projected/73ec613f-7f12-460f-8584-9c39f969a8e1-kube-api-access-lwt6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5rr86\" (UID: \"73ec613f-7f12-460f-8584-9c39f969a8e1\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.404688 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.404619    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:08:57.539522 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.539496    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"]
Jul 29 01:08:57.541887 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:08:57.541854    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ec613f_7f12_460f_8584_9c39f969a8e1.slice/crio-494ff73ad9faef4f876e48976fd469be73ba3af7b8d3f6b96899401cbd373d76 WatchSource:0}: Error finding container 494ff73ad9faef4f876e48976fd469be73ba3af7b8d3f6b96899401cbd373d76: Status 404 returned error can't find the container with id 494ff73ad9faef4f876e48976fd469be73ba3af7b8d3f6b96899401cbd373d76
Jul 29 01:08:57.660953 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:57.660866    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86" event={"ID":"73ec613f-7f12-460f-8584-9c39f969a8e1","Type":"ContainerStarted","Data":"494ff73ad9faef4f876e48976fd469be73ba3af7b8d3f6b96899401cbd373d76"}
Jul 29 01:08:58.667481 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:58.667441    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86" event={"ID":"73ec613f-7f12-460f-8584-9c39f969a8e1","Type":"ContainerStarted","Data":"693ba070ab09d0e8d5a9b49e92ed7efbd5c2265f2108e21f523bb50f0d29077c"}
Jul 29 01:08:58.685895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:08:58.685834    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86" podStartSLOduration=1.500076384 podStartE2EDuration="1.685818561s" totalImagesPullingTime="185.742177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:08:58.682965421 +0000 UTC m=+868.013615025" watchObservedRunningTime="2026-07-29 01:08:58.685818561 +0000 UTC m=+868.016468167"
Jul 29 01:09:07.405417 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:09:07.405377    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:09:07.405848 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:09:07.405445    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:09:07.418121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:09:07.418093    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:09:07.727413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:09:07.727384    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5rr86"
Jul 29 01:09:16.421879 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:09:16.421847    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:13.506238 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:13.506208    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:14.579469 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.579434    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:14.657666 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.657634    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:14.685833 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.685803    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bcdc844c4-j4njv"]
Jul 29 01:10:14.686014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.685978    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-bcdc844c4-j4njv" podUID="07030d7b-cf5d-41f8-98cd-5af28e1413f0" containerName="authorino" containerID="cri-o://00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f" gracePeriod=30
Jul 29 01:10:14.934635 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.934606    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:10:14.999407 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.999381    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07030d7b-cf5d-41f8-98cd-5af28e1413f0-tls-cert\") pod \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") "
Jul 29 01:10:14.999547 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:14.999450    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss5s7\" (UniqueName: \"kubernetes.io/projected/07030d7b-cf5d-41f8-98cd-5af28e1413f0-kube-api-access-ss5s7\") pod \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\" (UID: \"07030d7b-cf5d-41f8-98cd-5af28e1413f0\") "
Jul 29 01:10:15.001312 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.001260    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07030d7b-cf5d-41f8-98cd-5af28e1413f0-kube-api-access-ss5s7" (OuterVolumeSpecName: "kube-api-access-ss5s7") pod "07030d7b-cf5d-41f8-98cd-5af28e1413f0" (UID: "07030d7b-cf5d-41f8-98cd-5af28e1413f0"). InnerVolumeSpecName "kube-api-access-ss5s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:10:15.009202 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.009179    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07030d7b-cf5d-41f8-98cd-5af28e1413f0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "07030d7b-cf5d-41f8-98cd-5af28e1413f0" (UID: "07030d7b-cf5d-41f8-98cd-5af28e1413f0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:10:15.042331 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.042291    2555 generic.go:356] "Generic (PLEG): container finished" podID="07030d7b-cf5d-41f8-98cd-5af28e1413f0" containerID="00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f" exitCode=0
Jul 29 01:10:15.042331 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.042331    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bcdc844c4-j4njv"
Jul 29 01:10:15.042515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.042389    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcdc844c4-j4njv" event={"ID":"07030d7b-cf5d-41f8-98cd-5af28e1413f0","Type":"ContainerDied","Data":"00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f"}
Jul 29 01:10:15.042515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.042438    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bcdc844c4-j4njv" event={"ID":"07030d7b-cf5d-41f8-98cd-5af28e1413f0","Type":"ContainerDied","Data":"40c96d2566c5bad09fce4eb34a802dda2faba508d8887050997a5c8b6c042be8"}
Jul 29 01:10:15.042515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.042460    2555 scope.go:117] "RemoveContainer" containerID="00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f"
Jul 29 01:10:15.051059 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.051043    2555 scope.go:117] "RemoveContainer" containerID="00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f"
Jul 29 01:10:15.051327 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:10:15.051310    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f\": container with ID starting with 00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f not found: ID does not exist" containerID="00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f"
Jul 29 01:10:15.051383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.051338    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f"} err="failed to get container status \"00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f\": rpc error: code = NotFound desc = could not find container \"00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f\": container with ID starting with 00ab88341d635e456ec6d6fe4635c52fa391d2efb0da7d858c396cbd9457529f not found: ID does not exist"
Jul 29 01:10:15.065590 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.065565    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bcdc844c4-j4njv"]
Jul 29 01:10:15.071928 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.071899    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bcdc844c4-j4njv"]
Jul 29 01:10:15.100996 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.100970    2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07030d7b-cf5d-41f8-98cd-5af28e1413f0-tls-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:10:15.101097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.101045    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss5s7\" (UniqueName: \"kubernetes.io/projected/07030d7b-cf5d-41f8-98cd-5af28e1413f0-kube-api-access-ss5s7\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:10:15.189158 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:15.189123    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07030d7b-cf5d-41f8-98cd-5af28e1413f0" path="/var/lib/kubelet/pods/07030d7b-cf5d-41f8-98cd-5af28e1413f0/volumes"
Jul 29 01:10:18.376645 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:18.376614    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:53.468696 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:53.468613    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:53.529730 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:53.529702    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:10:59.976220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:59.975935    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d584db87-bmsh4"]
Jul 29 01:10:59.976220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:10:59.976202    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d584db87-bmsh4" podUID="5dfc19f8-af3f-4575-a586-7aa2aa672130" containerName="manager" containerID="cri-o://c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79" gracePeriod=30
Jul 29 01:11:00.228014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.227952    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:11:00.279114 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.279072    2555 generic.go:356] "Generic (PLEG): container finished" podID="5dfc19f8-af3f-4575-a586-7aa2aa672130" containerID="c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79" exitCode=0
Jul 29 01:11:00.279308 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.279120    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-bmsh4" event={"ID":"5dfc19f8-af3f-4575-a586-7aa2aa672130","Type":"ContainerDied","Data":"c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79"}
Jul 29 01:11:00.279308 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.279138    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-bmsh4"
Jul 29 01:11:00.279308 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.279212    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-bmsh4" event={"ID":"5dfc19f8-af3f-4575-a586-7aa2aa672130","Type":"ContainerDied","Data":"aef528f32b2cb498b0564fa8bc33c8548949f24d83336b257b17632ddb268244"}
Jul 29 01:11:00.279308 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.279242    2555 scope.go:117] "RemoveContainer" containerID="c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79"
Jul 29 01:11:00.287941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.287911    2555 scope.go:117] "RemoveContainer" containerID="c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79"
Jul 29 01:11:00.288178 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:11:00.288155    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79\": container with ID starting with c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79 not found: ID does not exist" containerID="c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79"
Jul 29 01:11:00.288343 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.288183    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79"} err="failed to get container status \"c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79\": rpc error: code = NotFound desc = could not find container \"c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79\": container with ID starting with c42f83d44201b27050267b6fdebc085578e6e5ba7dd9204f1a743704ea78ad79 not found: ID does not exist"
Jul 29 01:11:00.301438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.301417    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfc19f8-af3f-4575-a586-7aa2aa672130-webhook-cert\") pod \"5dfc19f8-af3f-4575-a586-7aa2aa672130\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") "
Jul 29 01:11:00.301549 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.301490    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gdff\" (UniqueName: \"kubernetes.io/projected/5dfc19f8-af3f-4575-a586-7aa2aa672130-kube-api-access-5gdff\") pod \"5dfc19f8-af3f-4575-a586-7aa2aa672130\" (UID: \"5dfc19f8-af3f-4575-a586-7aa2aa672130\") "
Jul 29 01:11:00.303306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.303265    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfc19f8-af3f-4575-a586-7aa2aa672130-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5dfc19f8-af3f-4575-a586-7aa2aa672130" (UID: "5dfc19f8-af3f-4575-a586-7aa2aa672130"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:11:00.303389 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.303322    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfc19f8-af3f-4575-a586-7aa2aa672130-kube-api-access-5gdff" (OuterVolumeSpecName: "kube-api-access-5gdff") pod "5dfc19f8-af3f-4575-a586-7aa2aa672130" (UID: "5dfc19f8-af3f-4575-a586-7aa2aa672130"). InnerVolumeSpecName "kube-api-access-5gdff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:11:00.403036 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.403007    2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dfc19f8-af3f-4575-a586-7aa2aa672130-webhook-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:11:00.403036 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.403032    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gdff\" (UniqueName: \"kubernetes.io/projected/5dfc19f8-af3f-4575-a586-7aa2aa672130-kube-api-access-5gdff\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:11:00.603325 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.603295    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d584db87-bmsh4"]
Jul 29 01:11:00.607222 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:00.607194    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d584db87-bmsh4"]
Jul 29 01:11:01.189724 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:01.189687    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dfc19f8-af3f-4575-a586-7aa2aa672130" path="/var/lib/kubelet/pods/5dfc19f8-af3f-4575-a586-7aa2aa672130/volumes"
Jul 29 01:11:07.880585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:07.880554    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:11:07.931199 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:11:07.931169    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:01.387199 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387159    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d584db87-lkqms"]
Jul 29 01:12:01.387642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387571    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dfc19f8-af3f-4575-a586-7aa2aa672130" containerName="manager"
Jul 29 01:12:01.387642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387585    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dfc19f8-af3f-4575-a586-7aa2aa672130" containerName="manager"
Jul 29 01:12:01.387642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387595    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07030d7b-cf5d-41f8-98cd-5af28e1413f0" containerName="authorino"
Jul 29 01:12:01.387642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387601    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="07030d7b-cf5d-41f8-98cd-5af28e1413f0" containerName="authorino"
Jul 29 01:12:01.387791 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387661    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dfc19f8-af3f-4575-a586-7aa2aa672130" containerName="manager"
Jul 29 01:12:01.387791 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.387672    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="07030d7b-cf5d-41f8-98cd-5af28e1413f0" containerName="authorino"
Jul 29 01:12:01.391185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.391162    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.393772 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.393747    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 01:12:01.395149 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.395127    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2w5vs\""
Jul 29 01:12:01.395268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.395184    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 01:12:01.401706 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.401677    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d584db87-lkqms"]
Jul 29 01:12:01.552183 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.552142    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52cc1988-8c09-450a-87e3-7096f05fdc4a-webhook-cert\") pod \"maas-controller-6d584db87-lkqms\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") " pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.552183 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.552186    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwglq\" (UniqueName: \"kubernetes.io/projected/52cc1988-8c09-450a-87e3-7096f05fdc4a-kube-api-access-bwglq\") pod \"maas-controller-6d584db87-lkqms\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") " pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.653288 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.653185    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52cc1988-8c09-450a-87e3-7096f05fdc4a-webhook-cert\") pod \"maas-controller-6d584db87-lkqms\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") " pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.653288 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.653228    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwglq\" (UniqueName: \"kubernetes.io/projected/52cc1988-8c09-450a-87e3-7096f05fdc4a-kube-api-access-bwglq\") pod \"maas-controller-6d584db87-lkqms\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") " pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.655691 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.655666    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52cc1988-8c09-450a-87e3-7096f05fdc4a-webhook-cert\") pod \"maas-controller-6d584db87-lkqms\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") " pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.661646 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.661613    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwglq\" (UniqueName: \"kubernetes.io/projected/52cc1988-8c09-450a-87e3-7096f05fdc4a-kube-api-access-bwglq\") pod \"maas-controller-6d584db87-lkqms\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") " pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.701959 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.701922    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:01.835098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.834910    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d584db87-lkqms"]
Jul 29 01:12:01.837867 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:12:01.837839    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52cc1988_8c09_450a_87e3_7096f05fdc4a.slice/crio-3b25d959b81657c27c3e18126533c02411294c120a8652299b4a46748b248330 WatchSource:0}: Error finding container 3b25d959b81657c27c3e18126533c02411294c120a8652299b4a46748b248330: Status 404 returned error can't find the container with id 3b25d959b81657c27c3e18126533c02411294c120a8652299b4a46748b248330
Jul 29 01:12:01.839179 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:01.839161    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:12:02.574450 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:02.574348    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-lkqms" event={"ID":"52cc1988-8c09-450a-87e3-7096f05fdc4a","Type":"ContainerStarted","Data":"6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc"}
Jul 29 01:12:02.574450 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:02.574406    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-lkqms" event={"ID":"52cc1988-8c09-450a-87e3-7096f05fdc4a","Type":"ContainerStarted","Data":"3b25d959b81657c27c3e18126533c02411294c120a8652299b4a46748b248330"}
Jul 29 01:12:02.574938 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:02.574468    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:02.592797 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:02.592723    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d584db87-lkqms" podStartSLOduration=1.210852818 podStartE2EDuration="1.592705597s" totalImagesPullingTime="381.852779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:02.592360064 +0000 UTC m=+1051.923009670" watchObservedRunningTime="2026-07-29 01:12:02.592705597 +0000 UTC m=+1051.923355203"
Jul 29 01:12:04.808456 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:04.808427    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:13.581554 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:13.581523    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:18.102327 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.102291    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:18.193100 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.193061    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d584db87-lkqms"]
Jul 29 01:12:18.193319 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.193299    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6d584db87-lkqms" podUID="52cc1988-8c09-450a-87e3-7096f05fdc4a" containerName="manager" containerID="cri-o://6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc" gracePeriod=30
Jul 29 01:12:18.441190 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.441165    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:18.505576 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.505544    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52cc1988-8c09-450a-87e3-7096f05fdc4a-webhook-cert\") pod \"52cc1988-8c09-450a-87e3-7096f05fdc4a\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") "
Jul 29 01:12:18.505760 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.505612    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwglq\" (UniqueName: \"kubernetes.io/projected/52cc1988-8c09-450a-87e3-7096f05fdc4a-kube-api-access-bwglq\") pod \"52cc1988-8c09-450a-87e3-7096f05fdc4a\" (UID: \"52cc1988-8c09-450a-87e3-7096f05fdc4a\") "
Jul 29 01:12:18.507595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.507565    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52cc1988-8c09-450a-87e3-7096f05fdc4a-kube-api-access-bwglq" (OuterVolumeSpecName: "kube-api-access-bwglq") pod "52cc1988-8c09-450a-87e3-7096f05fdc4a" (UID: "52cc1988-8c09-450a-87e3-7096f05fdc4a"). InnerVolumeSpecName "kube-api-access-bwglq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:12:18.507595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.507581    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52cc1988-8c09-450a-87e3-7096f05fdc4a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "52cc1988-8c09-450a-87e3-7096f05fdc4a" (UID: "52cc1988-8c09-450a-87e3-7096f05fdc4a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:12:18.606809 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.606767    2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52cc1988-8c09-450a-87e3-7096f05fdc4a-webhook-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:12:18.606809 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.606805    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwglq\" (UniqueName: \"kubernetes.io/projected/52cc1988-8c09-450a-87e3-7096f05fdc4a-kube-api-access-bwglq\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:12:18.654201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.654128    2555 generic.go:356] "Generic (PLEG): container finished" podID="52cc1988-8c09-450a-87e3-7096f05fdc4a" containerID="6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc" exitCode=0
Jul 29 01:12:18.654201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.654186    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-lkqms" event={"ID":"52cc1988-8c09-450a-87e3-7096f05fdc4a","Type":"ContainerDied","Data":"6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc"}
Jul 29 01:12:18.654201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.654194    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-lkqms"
Jul 29 01:12:18.654402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.654270    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-lkqms" event={"ID":"52cc1988-8c09-450a-87e3-7096f05fdc4a","Type":"ContainerDied","Data":"3b25d959b81657c27c3e18126533c02411294c120a8652299b4a46748b248330"}
Jul 29 01:12:18.654402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.654317    2555 scope.go:117] "RemoveContainer" containerID="6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc"
Jul 29 01:12:18.663623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.663604    2555 scope.go:117] "RemoveContainer" containerID="6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc"
Jul 29 01:12:18.663923 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:12:18.663903    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc\": container with ID starting with 6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc not found: ID does not exist" containerID="6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc"
Jul 29 01:12:18.663993 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.663930    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc"} err="failed to get container status \"6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc\": rpc error: code = NotFound desc = could not find container \"6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc\": container with ID starting with 6154bddc2262cb49104ad52c352788b5cf027df9ff06bdc409420c3934b94fdc not found: ID does not exist"
Jul 29 01:12:18.674157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.674125    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d584db87-lkqms"]
Jul 29 01:12:18.678353 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:18.678331    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d584db87-lkqms"]
Jul 29 01:12:19.190352 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.190323    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52cc1988-8c09-450a-87e3-7096f05fdc4a" path="/var/lib/kubelet/pods/52cc1988-8c09-450a-87e3-7096f05fdc4a/volumes"
Jul 29 01:12:19.306815 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.306781    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:19.410780 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.410748    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d584db87-4pl5c"]
Jul 29 01:12:19.411138 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.411126    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52cc1988-8c09-450a-87e3-7096f05fdc4a" containerName="manager"
Jul 29 01:12:19.411182 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.411139    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="52cc1988-8c09-450a-87e3-7096f05fdc4a" containerName="manager"
Jul 29 01:12:19.411215 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.411202    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="52cc1988-8c09-450a-87e3-7096f05fdc4a" containerName="manager"
Jul 29 01:12:19.415592 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.415574    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.419215 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.419193    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 01:12:19.419380 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.419325    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 01:12:19.419380 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.419333    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2w5vs\""
Jul 29 01:12:19.421761 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.421737    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d584db87-4pl5c"]
Jul 29 01:12:19.516220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.516128    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b8nb\" (UniqueName: \"kubernetes.io/projected/f7312171-669b-4145-b5f7-5eb924d8d14b-kube-api-access-9b8nb\") pod \"maas-controller-6d584db87-4pl5c\" (UID: \"f7312171-669b-4145-b5f7-5eb924d8d14b\") " pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.516220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.516165    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7312171-669b-4145-b5f7-5eb924d8d14b-webhook-cert\") pod \"maas-controller-6d584db87-4pl5c\" (UID: \"f7312171-669b-4145-b5f7-5eb924d8d14b\") " pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.617144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.617100    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b8nb\" (UniqueName: \"kubernetes.io/projected/f7312171-669b-4145-b5f7-5eb924d8d14b-kube-api-access-9b8nb\") pod \"maas-controller-6d584db87-4pl5c\" (UID: \"f7312171-669b-4145-b5f7-5eb924d8d14b\") " pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.617367 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.617153    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7312171-669b-4145-b5f7-5eb924d8d14b-webhook-cert\") pod \"maas-controller-6d584db87-4pl5c\" (UID: \"f7312171-669b-4145-b5f7-5eb924d8d14b\") " pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.619685 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.619665    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7312171-669b-4145-b5f7-5eb924d8d14b-webhook-cert\") pod \"maas-controller-6d584db87-4pl5c\" (UID: \"f7312171-669b-4145-b5f7-5eb924d8d14b\") " pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.626626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.626597    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b8nb\" (UniqueName: \"kubernetes.io/projected/f7312171-669b-4145-b5f7-5eb924d8d14b-kube-api-access-9b8nb\") pod \"maas-controller-6d584db87-4pl5c\" (UID: \"f7312171-669b-4145-b5f7-5eb924d8d14b\") " pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.727543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.727496    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:19.858982 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:19.858939    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d584db87-4pl5c"]
Jul 29 01:12:19.863915 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:12:19.863891    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7312171_669b_4145_b5f7_5eb924d8d14b.slice/crio-b0ef9e62d585e38d772b505b2dca753446744ef61451ccb63c20c2a185ed79c2 WatchSource:0}: Error finding container b0ef9e62d585e38d772b505b2dca753446744ef61451ccb63c20c2a185ed79c2: Status 404 returned error can't find the container with id b0ef9e62d585e38d772b505b2dca753446744ef61451ccb63c20c2a185ed79c2
Jul 29 01:12:20.667625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:20.667534    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-4pl5c" event={"ID":"f7312171-669b-4145-b5f7-5eb924d8d14b","Type":"ContainerStarted","Data":"b100c9d44a97ed6f860c361b1e4b55592f311be30e475faf2f48dc5dc1e42b68"}
Jul 29 01:12:20.667625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:20.667597    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d584db87-4pl5c" event={"ID":"f7312171-669b-4145-b5f7-5eb924d8d14b","Type":"ContainerStarted","Data":"b0ef9e62d585e38d772b505b2dca753446744ef61451ccb63c20c2a185ed79c2"}
Jul 29 01:12:20.668026 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:20.667632    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:20.685294 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:20.685229    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d584db87-4pl5c" podStartSLOduration=1.216309086 podStartE2EDuration="1.685207838s" totalImagesPullingTime="468.898752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:12:20.683303209 +0000 UTC m=+1070.013952830" watchObservedRunningTime="2026-07-29 01:12:20.685207838 +0000 UTC m=+1070.015857445"
Jul 29 01:12:31.674475 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:31.674389    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d584db87-4pl5c"
Jul 29 01:12:53.513449 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:53.513409    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:53.609189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:53.609153    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:53.714744 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:53.714710    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:59.431411 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:59.431373    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:59.534560 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:59.534527    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:59.632915 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:59.632877    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:12:59.779226 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:12:59.779195    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:13:05.432157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:13:05.432125    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:13:05.584292 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:13:05.584228    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:14:23.697766 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:14:23.697667    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:15:00.149220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.149177    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"]
Jul 29 01:15:00.152881 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.152860    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"
Jul 29 01:15:00.155594 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.155570    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:15:00.162707 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.162688    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"
Jul 29 01:15:00.168131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.168109    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"]
Jul 29 01:15:00.296310 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.296261    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"]
Jul 29 01:15:00.300397 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:15:00.300371    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8eeec9_e4ea_45ba_83d6_128500bf86f5.slice/crio-544fcd3c65118489bedab7f7700f352aafb5b0e107a7c689f3472fbcc31a07a2 WatchSource:0}: Error finding container 544fcd3c65118489bedab7f7700f352aafb5b0e107a7c689f3472fbcc31a07a2: Status 404 returned error can't find the container with id 544fcd3c65118489bedab7f7700f352aafb5b0e107a7c689f3472fbcc31a07a2
Jul 29 01:15:00.452420 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:00.452380    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj" event={"ID":"af8eeec9-e4ea-45ba-83d6-128500bf86f5","Type":"ContainerStarted","Data":"544fcd3c65118489bedab7f7700f352aafb5b0e107a7c689f3472fbcc31a07a2"}
Jul 29 01:15:03.470162 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:03.470124    2555 generic.go:356] "Generic (PLEG): container finished" podID="af8eeec9-e4ea-45ba-83d6-128500bf86f5" containerID="a6603c82b8cd738bbca7436843f87cbe5a4107ffda55a95fab1dc9b4dbece47b" exitCode=0
Jul 29 01:15:03.470546 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:03.470209    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj" event={"ID":"af8eeec9-e4ea-45ba-83d6-128500bf86f5","Type":"ContainerDied","Data":"a6603c82b8cd738bbca7436843f87cbe5a4107ffda55a95fab1dc9b4dbece47b"}
Jul 29 01:15:04.617069 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:04.617041    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"
Jul 29 01:15:05.481608 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:05.481526    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj"
Jul 29 01:15:05.481608 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:05.481546    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754795-6smqj" event={"ID":"af8eeec9-e4ea-45ba-83d6-128500bf86f5","Type":"ContainerDied","Data":"544fcd3c65118489bedab7f7700f352aafb5b0e107a7c689f3472fbcc31a07a2"}
Jul 29 01:15:05.481608 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:15:05.481589    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="544fcd3c65118489bedab7f7700f352aafb5b0e107a7c689f3472fbcc31a07a2"
Jul 29 01:17:04.965631 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:04.965558    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:17:06.349490 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:06.349454    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:17:06.410014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:06.409984    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:17:12.415310 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:12.415259    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"]
Jul 29 01:17:12.415780 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:12.415538    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2" podUID="000eebd0-2113-4a1b-96ff-97e13428a192" containerName="manager" containerID="cri-o://efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745" gracePeriod=10
Jul 29 01:17:15.479796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.479767    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:17:15.535776 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.535691    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/000eebd0-2113-4a1b-96ff-97e13428a192-extensions-socket-volume\") pod \"000eebd0-2113-4a1b-96ff-97e13428a192\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") "
Jul 29 01:17:15.535776 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.535743    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lktdt\" (UniqueName: \"kubernetes.io/projected/000eebd0-2113-4a1b-96ff-97e13428a192-kube-api-access-lktdt\") pod \"000eebd0-2113-4a1b-96ff-97e13428a192\" (UID: \"000eebd0-2113-4a1b-96ff-97e13428a192\") "
Jul 29 01:17:15.536175 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.536141    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/000eebd0-2113-4a1b-96ff-97e13428a192-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "000eebd0-2113-4a1b-96ff-97e13428a192" (UID: "000eebd0-2113-4a1b-96ff-97e13428a192"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:17:15.537843 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.537814    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/000eebd0-2113-4a1b-96ff-97e13428a192-kube-api-access-lktdt" (OuterVolumeSpecName: "kube-api-access-lktdt") pod "000eebd0-2113-4a1b-96ff-97e13428a192" (UID: "000eebd0-2113-4a1b-96ff-97e13428a192"). InnerVolumeSpecName "kube-api-access-lktdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:17:15.636580 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.636550    2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/000eebd0-2113-4a1b-96ff-97e13428a192-extensions-socket-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:17:15.636580 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:15.636577    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lktdt\" (UniqueName: \"kubernetes.io/projected/000eebd0-2113-4a1b-96ff-97e13428a192-kube-api-access-lktdt\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:17:16.110062 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.110027    2555 generic.go:356] "Generic (PLEG): container finished" podID="000eebd0-2113-4a1b-96ff-97e13428a192" containerID="efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745" exitCode=0
Jul 29 01:17:16.110251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.110078    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"
Jul 29 01:17:16.110251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.110109    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2" event={"ID":"000eebd0-2113-4a1b-96ff-97e13428a192","Type":"ContainerDied","Data":"efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745"}
Jul 29 01:17:16.110251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.110166    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2" event={"ID":"000eebd0-2113-4a1b-96ff-97e13428a192","Type":"ContainerDied","Data":"e2f1263b1d269a4d1265b03292a8cb9160b3dcaf5ccecdbed0b04c58646ade34"}
Jul 29 01:17:16.110251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.110190    2555 scope.go:117] "RemoveContainer" containerID="efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745"
Jul 29 01:17:16.120224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.120210    2555 scope.go:117] "RemoveContainer" containerID="efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745"
Jul 29 01:17:16.120504 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:17:16.120477    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745\": container with ID starting with efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745 not found: ID does not exist" containerID="efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745"
Jul 29 01:17:16.120555 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.120513    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745"} err="failed to get container status \"efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745\": rpc error: code = NotFound desc = could not find container \"efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745\": container with ID starting with efea7a542e228b110c6c7cb47a3a5accae626cca150965bacaf30cff35b02745 not found: ID does not exist"
Jul 29 01:17:16.132271 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.132250    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"]
Jul 29 01:17:16.136233 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:16.136208    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nvhs2"]
Jul 29 01:17:17.189950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:17:17.189909    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="000eebd0-2113-4a1b-96ff-97e13428a192" path="/var/lib/kubelet/pods/000eebd0-2113-4a1b-96ff-97e13428a192/volumes"
Jul 29 01:18:13.537350 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.537307    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"]
Jul 29 01:18:13.537928 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.537907    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="000eebd0-2113-4a1b-96ff-97e13428a192" containerName="manager"
Jul 29 01:18:13.537990 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.537930    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="000eebd0-2113-4a1b-96ff-97e13428a192" containerName="manager"
Jul 29 01:18:13.537990 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.537957    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8eeec9-e4ea-45ba-83d6-128500bf86f5" containerName="cleanup"
Jul 29 01:18:13.537990 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.537963    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8eeec9-e4ea-45ba-83d6-128500bf86f5" containerName="cleanup"
Jul 29 01:18:13.538098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.538025    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="af8eeec9-e4ea-45ba-83d6-128500bf86f5" containerName="cleanup"
Jul 29 01:18:13.538098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.538032    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="000eebd0-2113-4a1b-96ff-97e13428a192" containerName="manager"
Jul 29 01:18:13.541055 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.541038    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.544264 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.544242    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-75x59\""
Jul 29 01:18:13.563692 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.563666    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"]
Jul 29 01:18:13.632663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.632624    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74eb6dd2-6d54-4b91-8da7-046dcb5784df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vkbcv\" (UID: \"74eb6dd2-6d54-4b91-8da7-046dcb5784df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.632853 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.632678    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrfdd\" (UniqueName: \"kubernetes.io/projected/74eb6dd2-6d54-4b91-8da7-046dcb5784df-kube-api-access-rrfdd\") pod \"kuadrant-operator-controller-manager-86878c769f-vkbcv\" (UID: \"74eb6dd2-6d54-4b91-8da7-046dcb5784df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.733595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.733557    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrfdd\" (UniqueName: \"kubernetes.io/projected/74eb6dd2-6d54-4b91-8da7-046dcb5784df-kube-api-access-rrfdd\") pod \"kuadrant-operator-controller-manager-86878c769f-vkbcv\" (UID: \"74eb6dd2-6d54-4b91-8da7-046dcb5784df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.733770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.733663    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74eb6dd2-6d54-4b91-8da7-046dcb5784df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vkbcv\" (UID: \"74eb6dd2-6d54-4b91-8da7-046dcb5784df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.734082 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.734062    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/74eb6dd2-6d54-4b91-8da7-046dcb5784df-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-vkbcv\" (UID: \"74eb6dd2-6d54-4b91-8da7-046dcb5784df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.742865 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.742843    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrfdd\" (UniqueName: \"kubernetes.io/projected/74eb6dd2-6d54-4b91-8da7-046dcb5784df-kube-api-access-rrfdd\") pod \"kuadrant-operator-controller-manager-86878c769f-vkbcv\" (UID: \"74eb6dd2-6d54-4b91-8da7-046dcb5784df\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.851045 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.850954    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:13.991495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:13.991466    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"]
Jul 29 01:18:13.993440 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:18:13.993408    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74eb6dd2_6d54_4b91_8da7_046dcb5784df.slice/crio-846e36dc217edf46fea80a8c0416d2f1f864215fd46dcba572539b76d5eada80 WatchSource:0}: Error finding container 846e36dc217edf46fea80a8c0416d2f1f864215fd46dcba572539b76d5eada80: Status 404 returned error can't find the container with id 846e36dc217edf46fea80a8c0416d2f1f864215fd46dcba572539b76d5eada80
Jul 29 01:18:14.384659 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:14.384621    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv" event={"ID":"74eb6dd2-6d54-4b91-8da7-046dcb5784df","Type":"ContainerStarted","Data":"d358b2e1e48b5eb06a64b193a07c59676b2437889f93647819a815e16a45e624"}
Jul 29 01:18:14.384852 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:14.384677    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv" event={"ID":"74eb6dd2-6d54-4b91-8da7-046dcb5784df","Type":"ContainerStarted","Data":"846e36dc217edf46fea80a8c0416d2f1f864215fd46dcba572539b76d5eada80"}
Jul 29 01:18:14.384852 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:14.384751    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:18:14.410624 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:14.410553    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv" podStartSLOduration=1.410536066 podStartE2EDuration="1.410536066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:18:14.408411157 +0000 UTC m=+1423.739060763" watchObservedRunningTime="2026-07-29 01:18:14.410536066 +0000 UTC m=+1423.741185672"
Jul 29 01:18:25.392560 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:18:25.392523    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-vkbcv"
Jul 29 01:19:11.765579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:19:11.765540    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:19:33.849177 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:19:33.849141    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:20:56.144698 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:20:56.144661    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:21:03.181192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.181159    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"]
Jul 29 01:21:03.184764 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.184741    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.202131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.202103    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-5f6d371c-kserve-self-signed-certs\""
Jul 29 01:21:03.222590 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.222558    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"]
Jul 29 01:21:03.235709 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.235676    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-dshm\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.235888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.235855    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-tmp-dir\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.235964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.235886    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5067d389-a13b-45e0-aa21-c06ba88bb232-tls-certs\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.235964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.235921    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-home\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.235964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.235952    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cllzv\" (UniqueName: \"kubernetes.io/projected/5067d389-a13b-45e0-aa21-c06ba88bb232-kube-api-access-cllzv\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.236106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.235982    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-model-cache\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337160 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337118    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-dshm\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337196    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-tmp-dir\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337217    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5067d389-a13b-45e0-aa21-c06ba88bb232-tls-certs\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337240    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-home\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337261    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cllzv\" (UniqueName: \"kubernetes.io/projected/5067d389-a13b-45e0-aa21-c06ba88bb232-kube-api-access-cllzv\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337592 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337427    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-model-cache\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337717 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337695    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-home\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337777 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337730    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-tmp-dir\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.337777 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.337767    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-model-cache\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.339451 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.339424    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-dshm\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.339682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.339655    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5067d389-a13b-45e0-aa21-c06ba88bb232-tls-certs\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.357533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.357507    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cllzv\" (UniqueName: \"kubernetes.io/projected/5067d389-a13b-45e0-aa21-c06ba88bb232-kube-api-access-cllzv\") pod \"e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") " pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.397774 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.397734    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:21:03.494349 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.494311    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:03.685909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:03.685883    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"]
Jul 29 01:21:03.688482 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:21:03.688454    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5067d389_a13b_45e0_aa21_c06ba88bb232.slice/crio-5d37920bce2c42f43cf8b7b191827da7a45276ed378d743b87b852c7341df376 WatchSource:0}: Error finding container 5d37920bce2c42f43cf8b7b191827da7a45276ed378d743b87b852c7341df376: Status 404 returned error can't find the container with id 5d37920bce2c42f43cf8b7b191827da7a45276ed378d743b87b852c7341df376
Jul 29 01:21:04.238816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:04.238778    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992" event={"ID":"5067d389-a13b-45e0-aa21-c06ba88bb232","Type":"ContainerStarted","Data":"cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38"}
Jul 29 01:21:04.239289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:04.238877    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992" event={"ID":"5067d389-a13b-45e0-aa21-c06ba88bb232","Type":"ContainerStarted","Data":"5d37920bce2c42f43cf8b7b191827da7a45276ed378d743b87b852c7341df376"}
Jul 29 01:21:04.301988 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:04.301940    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992" podStartSLOduration=1.301925107 podStartE2EDuration="1.301925107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:21:04.299014916 +0000 UTC m=+1593.629664527" watchObservedRunningTime="2026-07-29 01:21:04.301925107 +0000 UTC m=+1593.632574712"
Jul 29 01:21:13.494880 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:13.494847    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:13.494880 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:13.494886    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:13.507460 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:13.507431    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:14.298253 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:14.298218    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:35.003833 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:35.003744    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:21:36.512533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:36.512491    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"]
Jul 29 01:21:36.527983 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:36.527947    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:21:51.532905 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.532868    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992" podUID="5067d389-a13b-45e0-aa21-c06ba88bb232" containerName="main" containerID="cri-o://cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38" gracePeriod=45
Jul 29 01:21:51.780001 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.779975    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:51.879501 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879421    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-dshm\") pod \"5067d389-a13b-45e0-aa21-c06ba88bb232\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") "
Jul 29 01:21:51.879663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879508    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-model-cache\") pod \"5067d389-a13b-45e0-aa21-c06ba88bb232\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") "
Jul 29 01:21:51.879663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879544    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-home\") pod \"5067d389-a13b-45e0-aa21-c06ba88bb232\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") "
Jul 29 01:21:51.879663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879567    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5067d389-a13b-45e0-aa21-c06ba88bb232-tls-certs\") pod \"5067d389-a13b-45e0-aa21-c06ba88bb232\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") "
Jul 29 01:21:51.879663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879594    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-tmp-dir\") pod \"5067d389-a13b-45e0-aa21-c06ba88bb232\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") "
Jul 29 01:21:51.879663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879618    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cllzv\" (UniqueName: \"kubernetes.io/projected/5067d389-a13b-45e0-aa21-c06ba88bb232-kube-api-access-cllzv\") pod \"5067d389-a13b-45e0-aa21-c06ba88bb232\" (UID: \"5067d389-a13b-45e0-aa21-c06ba88bb232\") "
Jul 29 01:21:51.879919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879816    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-model-cache" (OuterVolumeSpecName: "model-cache") pod "5067d389-a13b-45e0-aa21-c06ba88bb232" (UID: "5067d389-a13b-45e0-aa21-c06ba88bb232"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:51.879919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879831    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-home" (OuterVolumeSpecName: "home") pod "5067d389-a13b-45e0-aa21-c06ba88bb232" (UID: "5067d389-a13b-45e0-aa21-c06ba88bb232"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:51.879919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.879824    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5067d389-a13b-45e0-aa21-c06ba88bb232" (UID: "5067d389-a13b-45e0-aa21-c06ba88bb232"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:51.884332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.881378    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-tmp-dir\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:21:51.884332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.881407    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-model-cache\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:21:51.884332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.881422    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-home\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:21:51.884332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.881992    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-dshm" (OuterVolumeSpecName: "dshm") pod "5067d389-a13b-45e0-aa21-c06ba88bb232" (UID: "5067d389-a13b-45e0-aa21-c06ba88bb232"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:21:51.884332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.882473    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5067d389-a13b-45e0-aa21-c06ba88bb232-kube-api-access-cllzv" (OuterVolumeSpecName: "kube-api-access-cllzv") pod "5067d389-a13b-45e0-aa21-c06ba88bb232" (UID: "5067d389-a13b-45e0-aa21-c06ba88bb232"). InnerVolumeSpecName "kube-api-access-cllzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:21:51.888434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.888406    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5067d389-a13b-45e0-aa21-c06ba88bb232-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5067d389-a13b-45e0-aa21-c06ba88bb232" (UID: "5067d389-a13b-45e0-aa21-c06ba88bb232"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:21:51.982058 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.982025    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5067d389-a13b-45e0-aa21-c06ba88bb232-dshm\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:21:51.982058 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.982053    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5067d389-a13b-45e0-aa21-c06ba88bb232-tls-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:21:51.982223 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:51.982068    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cllzv\" (UniqueName: \"kubernetes.io/projected/5067d389-a13b-45e0-aa21-c06ba88bb232-kube-api-access-cllzv\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:21:52.476160 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.476122    2555 generic.go:356] "Generic (PLEG): container finished" podID="5067d389-a13b-45e0-aa21-c06ba88bb232" containerID="cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38" exitCode=0
Jul 29 01:21:52.476374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.476185    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"
Jul 29 01:21:52.476374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.476204    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992" event={"ID":"5067d389-a13b-45e0-aa21-c06ba88bb232","Type":"ContainerDied","Data":"cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38"}
Jul 29 01:21:52.476374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.476245    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992" event={"ID":"5067d389-a13b-45e0-aa21-c06ba88bb232","Type":"ContainerDied","Data":"5d37920bce2c42f43cf8b7b191827da7a45276ed378d743b87b852c7341df376"}
Jul 29 01:21:52.476374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.476262    2555 scope.go:117] "RemoveContainer" containerID="cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38"
Jul 29 01:21:52.485521 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.485505    2555 scope.go:117] "RemoveContainer" containerID="cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38"
Jul 29 01:21:52.485774 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:21:52.485754    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38\": container with ID starting with cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38 not found: ID does not exist" containerID="cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38"
Jul 29 01:21:52.485821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.485781    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38"} err="failed to get container status \"cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38\": rpc error: code = NotFound desc = could not find container \"cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38\": container with ID starting with cfcc983c3cb8525f616d5a67fba164e1656e0f2c0f80bf3759954f92a175dd38 not found: ID does not exist"
Jul 29 01:21:52.498715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.498685    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"]
Jul 29 01:21:52.503039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:52.503012    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-5f6d371c-kserve-7fc58bb46c-m4992"]
Jul 29 01:21:53.189391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:21:53.189349    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5067d389-a13b-45e0-aa21-c06ba88bb232" path="/var/lib/kubelet/pods/5067d389-a13b-45e0-aa21-c06ba88bb232/volumes"
Jul 29 01:23:31.809172 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:23:31.809135    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:24:48.284606 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:24:48.284571    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:09.778788 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:09.778752    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:12.708910 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:12.708875    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:12.807419 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:12.807385    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:13.149853 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:13.149775    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:19.282484 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:19.282439    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:19.355838 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:19.355809    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:19.506939 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:19.506908    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:19.819538 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:19.819490    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:19.876899 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:19.876869    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:21.808687 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.808649    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:21.914810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.914769    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"]
Jul 29 01:25:21.915213 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.915199    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5067d389-a13b-45e0-aa21-c06ba88bb232" containerName="main"
Jul 29 01:25:21.915267 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.915215    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5067d389-a13b-45e0-aa21-c06ba88bb232" containerName="main"
Jul 29 01:25:21.915328 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.915300    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5067d389-a13b-45e0-aa21-c06ba88bb232" containerName="main"
Jul 29 01:25:21.918848 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.918827    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:21.922893 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.922873    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bb686069-serving-cert\""
Jul 29 01:25:21.933293 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.933251    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:21.933586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:21.933558    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"]
Jul 29 01:25:22.027514 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.027482    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:22.056735 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.056703    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-maas-api-tls\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.056907 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.056741    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcfsw\" (UniqueName: \"kubernetes.io/projected/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-kube-api-access-bcfsw\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.056907 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.056770    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-openshift-service-ca\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.124926 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.124843    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:22.157358 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.157325    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-maas-api-tls\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.157358 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.157361    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcfsw\" (UniqueName: \"kubernetes.io/projected/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-kube-api-access-bcfsw\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.157358 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.157383    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-openshift-service-ca\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.158122 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.158101    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-openshift-service-ca\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.159892 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.159873    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-maas-api-tls\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.167785 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.167760    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcfsw\" (UniqueName: \"kubernetes.io/projected/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-kube-api-access-bcfsw\") pod \"maas-api-e2e-ait-bb686069-74455c955f-pcshx\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.230029 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.229994    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:22.373065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.372966    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"]
Jul 29 01:25:22.375426 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:25:22.375365    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fc3dd1_87c7_40aa_b6eb_7bf89c44f40a.slice/crio-e3cdd2cea1e765ec837a2fcd3a49a69c0c2fb41a4dbe9384f20dd6b340f6685f WatchSource:0}: Error finding container e3cdd2cea1e765ec837a2fcd3a49a69c0c2fb41a4dbe9384f20dd6b340f6685f: Status 404 returned error can't find the container with id e3cdd2cea1e765ec837a2fcd3a49a69c0c2fb41a4dbe9384f20dd6b340f6685f
Jul 29 01:25:22.376829 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.376810    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:25:22.505184 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.505144    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx" event={"ID":"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a","Type":"ContainerStarted","Data":"e3cdd2cea1e765ec837a2fcd3a49a69c0c2fb41a4dbe9384f20dd6b340f6685f"}
Jul 29 01:25:22.539219 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.539184    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:22.648530 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.648457    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:25:22.662083 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.662057    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"]
Jul 29 01:25:22.666882 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.666860    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.668991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.668968    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"]
Jul 29 01:25:22.669813 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.669791    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bb686069\""
Jul 29 01:25:22.672374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.672358    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.681087 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.681037    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"]
Jul 29 01:25:22.691925 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.691901    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"]
Jul 29 01:25:22.762361 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.762322    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdqs\" (UniqueName: \"kubernetes.io/projected/2f54eba7-d737-4865-add4-29c320fca3a3-kube-api-access-9hdqs\") pod \"payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.762536 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.762378    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f54eba7-d737-4865-add4-29c320fca3a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.863073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.863039    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592a548a-ad01-48db-90bf-d2b1c04da7dc-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") " pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.863073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.863077    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tn6v\" (UniqueName: \"kubernetes.io/projected/592a548a-ad01-48db-90bf-d2b1c04da7dc-kube-api-access-5tn6v\") pod \"payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") " pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.863553 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.863101    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdqs\" (UniqueName: \"kubernetes.io/projected/2f54eba7-d737-4865-add4-29c320fca3a3-kube-api-access-9hdqs\") pod \"payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.863553 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.863201    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f54eba7-d737-4865-add4-29c320fca3a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.863886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.863869    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f54eba7-d737-4865-add4-29c320fca3a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.874118 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.874099    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdqs\" (UniqueName: \"kubernetes.io/projected/2f54eba7-d737-4865-add4-29c320fca3a3-kube-api-access-9hdqs\") pod \"payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.964741 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.964711    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592a548a-ad01-48db-90bf-d2b1c04da7dc-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") " pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.964863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.964761    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tn6v\" (UniqueName: \"kubernetes.io/projected/592a548a-ad01-48db-90bf-d2b1c04da7dc-kube-api-access-5tn6v\") pod \"payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") " pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.965785 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.965756    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592a548a-ad01-48db-90bf-d2b1c04da7dc-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") " pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.973644 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.973624    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tn6v\" (UniqueName: \"kubernetes.io/projected/592a548a-ad01-48db-90bf-d2b1c04da7dc-kube-api-access-5tn6v\") pod \"payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") " pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:22.979585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.979557    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:22.988009 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:22.987988    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:23.342142 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.342111    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"]
Jul 29 01:25:23.343742 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:25:23.343710    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f54eba7_d737_4865_add4_29c320fca3a3.slice/crio-d96df243f06545a72735e39250dd69fe458240aec236ba68263cd3111768a622 WatchSource:0}: Error finding container d96df243f06545a72735e39250dd69fe458240aec236ba68263cd3111768a622: Status 404 returned error can't find the container with id d96df243f06545a72735e39250dd69fe458240aec236ba68263cd3111768a622
Jul 29 01:25:23.366615 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.366594    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"]
Jul 29 01:25:23.368700 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:25:23.368678    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592a548a_ad01_48db_90bf_d2b1c04da7dc.slice/crio-509cec2044d6a87aa66d0f190b4b4616ead76690fa137074f75c5c104fd231bb WatchSource:0}: Error finding container 509cec2044d6a87aa66d0f190b4b4616ead76690fa137074f75c5c104fd231bb: Status 404 returned error can't find the container with id 509cec2044d6a87aa66d0f190b4b4616ead76690fa137074f75c5c104fd231bb
Jul 29 01:25:23.511980 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.511946    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt" event={"ID":"2f54eba7-d737-4865-add4-29c320fca3a3","Type":"ContainerStarted","Data":"271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e"}
Jul 29 01:25:23.512168 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.511997    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt" event={"ID":"2f54eba7-d737-4865-add4-29c320fca3a3","Type":"ContainerStarted","Data":"d96df243f06545a72735e39250dd69fe458240aec236ba68263cd3111768a622"}
Jul 29 01:25:23.512168 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.512031    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:25:23.513712 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.513689    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx" event={"ID":"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a","Type":"ContainerStarted","Data":"e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3"}
Jul 29 01:25:23.513841 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.513774    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:23.515231 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.515207    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm" event={"ID":"592a548a-ad01-48db-90bf-d2b1c04da7dc","Type":"ContainerStarted","Data":"e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5"}
Jul 29 01:25:23.515357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.515267    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm" event={"ID":"592a548a-ad01-48db-90bf-d2b1c04da7dc","Type":"ContainerStarted","Data":"509cec2044d6a87aa66d0f190b4b4616ead76690fa137074f75c5c104fd231bb"}
Jul 29 01:25:23.515357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.515323    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:23.532402 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.532351    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt" podStartSLOduration=1.53233362 podStartE2EDuration="1.53233362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:25:23.530449326 +0000 UTC m=+1852.861098933" watchObservedRunningTime="2026-07-29 01:25:23.53233362 +0000 UTC m=+1852.862983227"
Jul 29 01:25:23.548115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.548059    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm" podStartSLOduration=1.548041255 podStartE2EDuration="1.548041255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:25:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:25:23.547379301 +0000 UTC m=+1852.878029105" watchObservedRunningTime="2026-07-29 01:25:23.548041255 +0000 UTC m=+1852.878690862"
Jul 29 01:25:23.566413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:23.566366    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx" podStartSLOduration=1.9969713059999998 podStartE2EDuration="2.566350113s" totalImagesPullingTime="569.378807ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:25:23.564048823 +0000 UTC m=+1852.894698431" watchObservedRunningTime="2026-07-29 01:25:23.566350113 +0000 UTC m=+1852.896999719"
Jul 29 01:25:29.525374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:29.525338    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:25:34.522049 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:34.522017    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:25:34.522501 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:25:34.522167    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:26:02.654107 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.654071    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"]
Jul 29 01:26:02.656959 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.654313    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx" podUID="b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" containerName="maas-api" containerID="cri-o://e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3" gracePeriod=30
Jul 29 01:26:02.673769 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.673744    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:02.703210 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.703171    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"]
Jul 29 01:26:02.703509 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.703455    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm" podUID="592a548a-ad01-48db-90bf-d2b1c04da7dc" containerName="payload-processing" containerID="cri-o://e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5" gracePeriod=30
Jul 29 01:26:02.713185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.713072    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"]
Jul 29 01:26:02.713409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.713346    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt" podUID="2f54eba7-d737-4865-add4-29c320fca3a3" containerName="payload-pre-processing" containerID="cri-o://271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e" gracePeriod=30
Jul 29 01:26:02.751123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.751090    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:02.812544 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.812506    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:02.887534 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.887190    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:02.970023 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:02.969709    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:26:03.003655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.003626    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:26:03.021209 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.021175    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:26:03.047960 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.047862    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:03.119231 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119200    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f54eba7-d737-4865-add4-29c320fca3a3-plugins-config-volume\") pod \"2f54eba7-d737-4865-add4-29c320fca3a3\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") "
Jul 29 01:26:03.119231 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119242    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tn6v\" (UniqueName: \"kubernetes.io/projected/592a548a-ad01-48db-90bf-d2b1c04da7dc-kube-api-access-5tn6v\") pod \"592a548a-ad01-48db-90bf-d2b1c04da7dc\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") "
Jul 29 01:26:03.119485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119267    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-maas-api-tls\") pod \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") "
Jul 29 01:26:03.119485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119330    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-openshift-service-ca\") pod \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") "
Jul 29 01:26:03.119485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119372    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hdqs\" (UniqueName: \"kubernetes.io/projected/2f54eba7-d737-4865-add4-29c320fca3a3-kube-api-access-9hdqs\") pod \"2f54eba7-d737-4865-add4-29c320fca3a3\" (UID: \"2f54eba7-d737-4865-add4-29c320fca3a3\") "
Jul 29 01:26:03.119485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119401    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592a548a-ad01-48db-90bf-d2b1c04da7dc-plugins-config-volume\") pod \"592a548a-ad01-48db-90bf-d2b1c04da7dc\" (UID: \"592a548a-ad01-48db-90bf-d2b1c04da7dc\") "
Jul 29 01:26:03.119485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119446    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcfsw\" (UniqueName: \"kubernetes.io/projected/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-kube-api-access-bcfsw\") pod \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\" (UID: \"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a\") "
Jul 29 01:26:03.119754 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119731    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" (UID: "b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:26:03.119843 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119811    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f54eba7-d737-4865-add4-29c320fca3a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f54eba7-d737-4865-add4-29c320fca3a3" (UID: "2f54eba7-d737-4865-add4-29c320fca3a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:26:03.119907 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.119857    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/592a548a-ad01-48db-90bf-d2b1c04da7dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "592a548a-ad01-48db-90bf-d2b1c04da7dc" (UID: "592a548a-ad01-48db-90bf-d2b1c04da7dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:26:03.121550 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.121523    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/592a548a-ad01-48db-90bf-d2b1c04da7dc-kube-api-access-5tn6v" (OuterVolumeSpecName: "kube-api-access-5tn6v") pod "592a548a-ad01-48db-90bf-d2b1c04da7dc" (UID: "592a548a-ad01-48db-90bf-d2b1c04da7dc"). InnerVolumeSpecName "kube-api-access-5tn6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:26:03.121692 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.121671    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" (UID: "b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:26:03.121921 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.121903    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f54eba7-d737-4865-add4-29c320fca3a3-kube-api-access-9hdqs" (OuterVolumeSpecName: "kube-api-access-9hdqs") pod "2f54eba7-d737-4865-add4-29c320fca3a3" (UID: "2f54eba7-d737-4865-add4-29c320fca3a3"). InnerVolumeSpecName "kube-api-access-9hdqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:26:03.121974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.121955    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-kube-api-access-bcfsw" (OuterVolumeSpecName: "kube-api-access-bcfsw") pod "b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" (UID: "b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a"). InnerVolumeSpecName "kube-api-access-bcfsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:26:03.220251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220219    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hdqs\" (UniqueName: \"kubernetes.io/projected/2f54eba7-d737-4865-add4-29c320fca3a3-kube-api-access-9hdqs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.220251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220247    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/592a548a-ad01-48db-90bf-d2b1c04da7dc-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.220425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220261    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcfsw\" (UniqueName: \"kubernetes.io/projected/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-kube-api-access-bcfsw\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.220425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220270    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f54eba7-d737-4865-add4-29c320fca3a3-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.220425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220293    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tn6v\" (UniqueName: \"kubernetes.io/projected/592a548a-ad01-48db-90bf-d2b1c04da7dc-kube-api-access-5tn6v\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.220425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220302    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.220425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.220311    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:03.263019 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.262989    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:03.510597 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.510521    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:03.610849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.610818    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:03.719139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.719106    2555 generic.go:356] "Generic (PLEG): container finished" podID="b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" containerID="e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3" exitCode=0
Jul 29 01:26:03.719615 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.719148    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"
Jul 29 01:26:03.719615 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.719187    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx" event={"ID":"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a","Type":"ContainerDied","Data":"e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3"}
Jul 29 01:26:03.719615 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.719238    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx" event={"ID":"b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a","Type":"ContainerDied","Data":"e3cdd2cea1e765ec837a2fcd3a49a69c0c2fb41a4dbe9384f20dd6b340f6685f"}
Jul 29 01:26:03.719615 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.719260    2555 scope.go:117] "RemoveContainer" containerID="e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3"
Jul 29 01:26:03.720602 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.720582    2555 generic.go:356] "Generic (PLEG): container finished" podID="592a548a-ad01-48db-90bf-d2b1c04da7dc" containerID="e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5" exitCode=0
Jul 29 01:26:03.720695 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.720621    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"
Jul 29 01:26:03.720695 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.720653    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm" event={"ID":"592a548a-ad01-48db-90bf-d2b1c04da7dc","Type":"ContainerDied","Data":"e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5"}
Jul 29 01:26:03.720802 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.720707    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm" event={"ID":"592a548a-ad01-48db-90bf-d2b1c04da7dc","Type":"ContainerDied","Data":"509cec2044d6a87aa66d0f190b4b4616ead76690fa137074f75c5c104fd231bb"}
Jul 29 01:26:03.722193 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.722169    2555 generic.go:356] "Generic (PLEG): container finished" podID="2f54eba7-d737-4865-add4-29c320fca3a3" containerID="271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e" exitCode=0
Jul 29 01:26:03.722319 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.722214    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt" event={"ID":"2f54eba7-d737-4865-add4-29c320fca3a3","Type":"ContainerDied","Data":"271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e"}
Jul 29 01:26:03.722319 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.722221    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"
Jul 29 01:26:03.722319 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.722252    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt" event={"ID":"2f54eba7-d737-4865-add4-29c320fca3a3","Type":"ContainerDied","Data":"d96df243f06545a72735e39250dd69fe458240aec236ba68263cd3111768a622"}
Jul 29 01:26:03.730694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.730593    2555 scope.go:117] "RemoveContainer" containerID="e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3"
Jul 29 01:26:03.731033 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:26:03.731008    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3\": container with ID starting with e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3 not found: ID does not exist" containerID="e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3"
Jul 29 01:26:03.731121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.731042    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3"} err="failed to get container status \"e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3\": rpc error: code = NotFound desc = could not find container \"e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3\": container with ID starting with e34673290b5bfcec1fe2ec7cf88901d4f867ceaa6ef2e9043ec3e0550cf284b3 not found: ID does not exist"
Jul 29 01:26:03.731121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.731060    2555 scope.go:117] "RemoveContainer" containerID="e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5"
Jul 29 01:26:03.735115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.735096    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:03.739553 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.739537    2555 scope.go:117] "RemoveContainer" containerID="e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5"
Jul 29 01:26:03.739791 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:26:03.739774    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5\": container with ID starting with e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5 not found: ID does not exist" containerID="e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5"
Jul 29 01:26:03.739851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.739798    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5"} err="failed to get container status \"e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5\": rpc error: code = NotFound desc = could not find container \"e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5\": container with ID starting with e5f5af02c3764476761fc2553c04233183e99929050230f47a338c48382902d5 not found: ID does not exist"
Jul 29 01:26:03.739851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.739814    2555 scope.go:117] "RemoveContainer" containerID="271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e"
Jul 29 01:26:03.744938 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.744917    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"]
Jul 29 01:26:03.748222 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.748181    2555 scope.go:117] "RemoveContainer" containerID="271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e"
Jul 29 01:26:03.748566 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:26:03.748532    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e\": container with ID starting with 271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e not found: ID does not exist" containerID="271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e"
Jul 29 01:26:03.748702 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.748574    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e"} err="failed to get container status \"271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e\": rpc error: code = NotFound desc = could not find container \"271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e\": container with ID starting with 271d5939a80b44b8ac3d34baf62d5b3098ffde9c6d5b2b12e8f54ba2c937e77e not found: ID does not exist"
Jul 29 01:26:03.751376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.751358    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb686069-764b885bcd-r9fbm"]
Jul 29 01:26:03.772385 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.772336    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"]
Jul 29 01:26:03.777674 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.777652    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb686069-74455c955f-pcshx"]
Jul 29 01:26:03.790460 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.790440    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"]
Jul 29 01:26:03.797926 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:03.797902    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb686069-cd9b566d-7dstt"]
Jul 29 01:26:04.105716 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:04.105632    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:04.364338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:04.364236    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:05.189115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:05.189073    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f54eba7-d737-4865-add4-29c320fca3a3" path="/var/lib/kubelet/pods/2f54eba7-d737-4865-add4-29c320fca3a3/volumes"
Jul 29 01:26:05.189655 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:05.189633    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="592a548a-ad01-48db-90bf-d2b1c04da7dc" path="/var/lib/kubelet/pods/592a548a-ad01-48db-90bf-d2b1c04da7dc/volumes"
Jul 29 01:26:05.190067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:05.190047    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" path="/var/lib/kubelet/pods/b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a/volumes"
Jul 29 01:26:06.199728 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.199680    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:06.394143 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394106    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"]
Jul 29 01:26:06.394691 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394642    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" containerName="maas-api"
Jul 29 01:26:06.394691 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394670    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" containerName="maas-api"
Jul 29 01:26:06.394691 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394695    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="592a548a-ad01-48db-90bf-d2b1c04da7dc" containerName="payload-processing"
Jul 29 01:26:06.394975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394703    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="592a548a-ad01-48db-90bf-d2b1c04da7dc" containerName="payload-processing"
Jul 29 01:26:06.394975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394719    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f54eba7-d737-4865-add4-29c320fca3a3" containerName="payload-pre-processing"
Jul 29 01:26:06.394975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394727    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f54eba7-d737-4865-add4-29c320fca3a3" containerName="payload-pre-processing"
Jul 29 01:26:06.394975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394849    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8fc3dd1-87c7-40aa-b6eb-7bf89c44f40a" containerName="maas-api"
Jul 29 01:26:06.394975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394864    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f54eba7-d737-4865-add4-29c320fca3a3" containerName="payload-pre-processing"
Jul 29 01:26:06.394975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.394878    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="592a548a-ad01-48db-90bf-d2b1c04da7dc" containerName="payload-processing"
Jul 29 01:26:06.399598 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.399576    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.404595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.404571    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-aaa0c308-serving-cert\""
Jul 29 01:26:06.413835 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.413812    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"]
Jul 29 01:26:06.439725 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.439683    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:06.548750 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.548671    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-openshift-service-ca\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.548893 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.548759    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cczvg\" (UniqueName: \"kubernetes.io/projected/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-kube-api-access-cczvg\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.548893 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.548800    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-maas-api-tls\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.595756 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.595724    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:06.649516 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.649481    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cczvg\" (UniqueName: \"kubernetes.io/projected/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-kube-api-access-cczvg\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.649699 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.649542    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-maas-api-tls\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.649699 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.649583    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-openshift-service-ca\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.650229 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.650203    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-openshift-service-ca\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.651986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.651955    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-maas-api-tls\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.659892 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.659872    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cczvg\" (UniqueName: \"kubernetes.io/projected/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-kube-api-access-cczvg\") pod \"maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.713820 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.713796    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:06.868820 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.868775    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"]
Jul 29 01:26:06.871409 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:26:06.871383    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ec2da9_0fcd_4c9c_88ee_356b662792c3.slice/crio-0caad87093e29ec6e5742eb9768c4d5147c33578b7bd9d359c4444f3219da84c WatchSource:0}: Error finding container 0caad87093e29ec6e5742eb9768c4d5147c33578b7bd9d359c4444f3219da84c: Status 404 returned error can't find the container with id 0caad87093e29ec6e5742eb9768c4d5147c33578b7bd9d359c4444f3219da84c
Jul 29 01:26:06.952663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:06.952635    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:07.049625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.049591    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:07.050816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.050796    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"]
Jul 29 01:26:07.055640 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.055621    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.058326 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.058310    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-aaa0c308\""
Jul 29 01:26:07.069482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.069427    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"]
Jul 29 01:26:07.078848 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.078826    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"]
Jul 29 01:26:07.082434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.082417    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.085077 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.085061    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-aaa0c308-dockercfg-whjth\""
Jul 29 01:26:07.097879 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.097860    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"]
Jul 29 01:26:07.156872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.156845    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7410469a-6dfe-471a-862f-322b49f96186-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.157028 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.156917    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grr8t\" (UniqueName: \"kubernetes.io/projected/7410469a-6dfe-471a-862f-322b49f96186-kube-api-access-grr8t\") pod \"payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.258239 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.258213    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grr8t\" (UniqueName: \"kubernetes.io/projected/7410469a-6dfe-471a-862f-322b49f96186-kube-api-access-grr8t\") pod \"payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.258576 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.258301    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7410469a-6dfe-471a-862f-322b49f96186-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.258576 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.258326    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br4z4\" (UniqueName: \"kubernetes.io/projected/75345eba-8039-408f-b25e-c4c64257ebbd-kube-api-access-br4z4\") pod \"payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") " pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.258576 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.258357    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75345eba-8039-408f-b25e-c4c64257ebbd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") " pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.258992 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.258974    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7410469a-6dfe-471a-862f-322b49f96186-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.269822 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.269794    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grr8t\" (UniqueName: \"kubernetes.io/projected/7410469a-6dfe-471a-862f-322b49f96186-kube-api-access-grr8t\") pod \"payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.359814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.359741    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br4z4\" (UniqueName: \"kubernetes.io/projected/75345eba-8039-408f-b25e-c4c64257ebbd-kube-api-access-br4z4\") pod \"payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") " pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.359814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.359800    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75345eba-8039-408f-b25e-c4c64257ebbd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") " pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.360614 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.360591    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75345eba-8039-408f-b25e-c4c64257ebbd-plugins-config-volume\") pod \"payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") " pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.366654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.366625    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.368900 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.368881    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br4z4\" (UniqueName: \"kubernetes.io/projected/75345eba-8039-408f-b25e-c4c64257ebbd-kube-api-access-br4z4\") pod \"payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") " pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.393555 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.393532    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:07.543629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.543603    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"]
Jul 29 01:26:07.545759 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:26:07.545734    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7410469a_6dfe_471a_862f_322b49f96186.slice/crio-b530a396e8edc93cc9abd46578921f8e54a323ade1ee206233ec7b81531a394f WatchSource:0}: Error finding container b530a396e8edc93cc9abd46578921f8e54a323ade1ee206233ec7b81531a394f: Status 404 returned error can't find the container with id b530a396e8edc93cc9abd46578921f8e54a323ade1ee206233ec7b81531a394f
Jul 29 01:26:07.749027 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.748996    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd" event={"ID":"7410469a-6dfe-471a-862f-322b49f96186","Type":"ContainerStarted","Data":"be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf"}
Jul 29 01:26:07.749211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.749049    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd" event={"ID":"7410469a-6dfe-471a-862f-322b49f96186","Type":"ContainerStarted","Data":"b530a396e8edc93cc9abd46578921f8e54a323ade1ee206233ec7b81531a394f"}
Jul 29 01:26:07.749211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.749080    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:07.750643 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.750620    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp" event={"ID":"c0ec2da9-0fcd-4c9c-88ee-356b662792c3","Type":"ContainerStarted","Data":"bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f"}
Jul 29 01:26:07.750763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.750659    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp" event={"ID":"c0ec2da9-0fcd-4c9c-88ee-356b662792c3","Type":"ContainerStarted","Data":"0caad87093e29ec6e5742eb9768c4d5147c33578b7bd9d359c4444f3219da84c"}
Jul 29 01:26:07.750763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.750685    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:07.754256 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.754221    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"]
Jul 29 01:26:07.757015 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:26:07.756990    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75345eba_8039_408f_b25e_c4c64257ebbd.slice/crio-0f04bca792d5e0e05576559463298adbba84efca939a0dc49ffbad71783b98cc WatchSource:0}: Error finding container 0f04bca792d5e0e05576559463298adbba84efca939a0dc49ffbad71783b98cc: Status 404 returned error can't find the container with id 0f04bca792d5e0e05576559463298adbba84efca939a0dc49ffbad71783b98cc
Jul 29 01:26:07.774463 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.774414    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd" podStartSLOduration=0.774398129 podStartE2EDuration="774.398129ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:07.772578248 +0000 UTC m=+1897.103227851" watchObservedRunningTime="2026-07-29 01:26:07.774398129 +0000 UTC m=+1897.105047735"
Jul 29 01:26:07.791474 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:07.791430    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp" podStartSLOduration=1.473536612 podStartE2EDuration="1.791414689s" totalImagesPullingTime="317.878077ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:07.79010433 +0000 UTC m=+1897.120753963" watchObservedRunningTime="2026-07-29 01:26:07.791414689 +0000 UTC m=+1897.122064325"
Jul 29 01:26:08.757187 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:08.757152    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq" event={"ID":"75345eba-8039-408f-b25e-c4c64257ebbd","Type":"ContainerStarted","Data":"f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43"}
Jul 29 01:26:08.757682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:08.757232    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq" event={"ID":"75345eba-8039-408f-b25e-c4c64257ebbd","Type":"ContainerStarted","Data":"0f04bca792d5e0e05576559463298adbba84efca939a0dc49ffbad71783b98cc"}
Jul 29 01:26:08.757682 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:08.757632    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:08.774866 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:08.774821    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq" podStartSLOduration=1.774807596 podStartE2EDuration="1.774807596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:08.772418438 +0000 UTC m=+1898.103068044" watchObservedRunningTime="2026-07-29 01:26:08.774807596 +0000 UTC m=+1898.105457205"
Jul 29 01:26:13.763292 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:13.763243    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:18.759775 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:18.759742    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:20.770790 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:20.770755    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:37.000516 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.000479    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"]
Jul 29 01:26:37.000931 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.000705    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp" podUID="c0ec2da9-0fcd-4c9c-88ee-356b662792c3" containerName="maas-api" containerID="cri-o://bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f" gracePeriod=30
Jul 29 01:26:37.025633 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.025604    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.057875 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.057838    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"]
Jul 29 01:26:37.058106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.058064    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq" podUID="75345eba-8039-408f-b25e-c4c64257ebbd" containerName="payload-processing" containerID="cri-o://f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43" gracePeriod=30
Jul 29 01:26:37.070001 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.069977    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"]
Jul 29 01:26:37.070476 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.070435    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd" podUID="7410469a-6dfe-471a-862f-322b49f96186" containerName="payload-pre-processing" containerID="cri-o://be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf" gracePeriod=30
Jul 29 01:26:37.101758 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.101726    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.174420 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.174385    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.307347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.307313    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.326867 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.326843    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:37.357700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.357672    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:37.380618 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.380594    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:37.432764 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.432729    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-openshift-service-ca\") pod \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") "
Jul 29 01:26:37.432966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.432796    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cczvg\" (UniqueName: \"kubernetes.io/projected/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-kube-api-access-cczvg\") pod \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") "
Jul 29 01:26:37.432966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.432907    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-maas-api-tls\") pod \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\" (UID: \"c0ec2da9-0fcd-4c9c-88ee-356b662792c3\") "
Jul 29 01:26:37.432966 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.432939    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75345eba-8039-408f-b25e-c4c64257ebbd-plugins-config-volume\") pod \"75345eba-8039-408f-b25e-c4c64257ebbd\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") "
Jul 29 01:26:37.433144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.432978    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-br4z4\" (UniqueName: \"kubernetes.io/projected/75345eba-8039-408f-b25e-c4c64257ebbd-kube-api-access-br4z4\") pod \"75345eba-8039-408f-b25e-c4c64257ebbd\" (UID: \"75345eba-8039-408f-b25e-c4c64257ebbd\") "
Jul 29 01:26:37.434403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.434366    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75345eba-8039-408f-b25e-c4c64257ebbd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75345eba-8039-408f-b25e-c4c64257ebbd" (UID: "75345eba-8039-408f-b25e-c4c64257ebbd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:26:37.434763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.434735    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0ec2da9-0fcd-4c9c-88ee-356b662792c3" (UID: "c0ec2da9-0fcd-4c9c-88ee-356b662792c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:26:37.435781 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.435755    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-kube-api-access-cczvg" (OuterVolumeSpecName: "kube-api-access-cczvg") pod "c0ec2da9-0fcd-4c9c-88ee-356b662792c3" (UID: "c0ec2da9-0fcd-4c9c-88ee-356b662792c3"). InnerVolumeSpecName "kube-api-access-cczvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:26:37.435979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.435953    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75345eba-8039-408f-b25e-c4c64257ebbd-kube-api-access-br4z4" (OuterVolumeSpecName: "kube-api-access-br4z4") pod "75345eba-8039-408f-b25e-c4c64257ebbd" (UID: "75345eba-8039-408f-b25e-c4c64257ebbd"). InnerVolumeSpecName "kube-api-access-br4z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:26:37.436085 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.436010    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0ec2da9-0fcd-4c9c-88ee-356b662792c3" (UID: "c0ec2da9-0fcd-4c9c-88ee-356b662792c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:26:37.456372 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.456330    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.533782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.533694    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7410469a-6dfe-471a-862f-322b49f96186-plugins-config-volume\") pod \"7410469a-6dfe-471a-862f-322b49f96186\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") "
Jul 29 01:26:37.533782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.533741    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grr8t\" (UniqueName: \"kubernetes.io/projected/7410469a-6dfe-471a-862f-322b49f96186-kube-api-access-grr8t\") pod \"7410469a-6dfe-471a-862f-322b49f96186\" (UID: \"7410469a-6dfe-471a-862f-322b49f96186\") "
Jul 29 01:26:37.534007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.533963    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cczvg\" (UniqueName: \"kubernetes.io/projected/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-kube-api-access-cczvg\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.534007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.533975    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.534007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.533984    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75345eba-8039-408f-b25e-c4c64257ebbd-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.534007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.533992    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br4z4\" (UniqueName: \"kubernetes.io/projected/75345eba-8039-408f-b25e-c4c64257ebbd-kube-api-access-br4z4\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.534007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.534003    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ec2da9-0fcd-4c9c-88ee-356b662792c3-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.534239 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.534148    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7410469a-6dfe-471a-862f-322b49f96186-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7410469a-6dfe-471a-862f-322b49f96186" (UID: "7410469a-6dfe-471a-862f-322b49f96186"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:26:37.535881 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.535854    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7410469a-6dfe-471a-862f-322b49f96186-kube-api-access-grr8t" (OuterVolumeSpecName: "kube-api-access-grr8t") pod "7410469a-6dfe-471a-862f-322b49f96186" (UID: "7410469a-6dfe-471a-862f-322b49f96186"). InnerVolumeSpecName "kube-api-access-grr8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:26:37.635289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.635236    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7410469a-6dfe-471a-862f-322b49f96186-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.635289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.635268    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grr8t\" (UniqueName: \"kubernetes.io/projected/7410469a-6dfe-471a-862f-322b49f96186-kube-api-access-grr8t\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:26:37.789673 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.789596    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.904781 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.904744    2555 generic.go:356] "Generic (PLEG): container finished" podID="c0ec2da9-0fcd-4c9c-88ee-356b662792c3" containerID="bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f" exitCode=0
Jul 29 01:26:37.904950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.904795    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"
Jul 29 01:26:37.904950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.904828    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp" event={"ID":"c0ec2da9-0fcd-4c9c-88ee-356b662792c3","Type":"ContainerDied","Data":"bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f"}
Jul 29 01:26:37.904950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.904893    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp" event={"ID":"c0ec2da9-0fcd-4c9c-88ee-356b662792c3","Type":"ContainerDied","Data":"0caad87093e29ec6e5742eb9768c4d5147c33578b7bd9d359c4444f3219da84c"}
Jul 29 01:26:37.904950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.904925    2555 scope.go:117] "RemoveContainer" containerID="bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f"
Jul 29 01:26:37.906098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.906068    2555 generic.go:356] "Generic (PLEG): container finished" podID="75345eba-8039-408f-b25e-c4c64257ebbd" containerID="f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43" exitCode=0
Jul 29 01:26:37.906184 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.906119    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"
Jul 29 01:26:37.906184 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.906146    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq" event={"ID":"75345eba-8039-408f-b25e-c4c64257ebbd","Type":"ContainerDied","Data":"f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43"}
Jul 29 01:26:37.906302 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.906193    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq" event={"ID":"75345eba-8039-408f-b25e-c4c64257ebbd","Type":"ContainerDied","Data":"0f04bca792d5e0e05576559463298adbba84efca939a0dc49ffbad71783b98cc"}
Jul 29 01:26:37.907648 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.907626    2555 generic.go:356] "Generic (PLEG): container finished" podID="7410469a-6dfe-471a-862f-322b49f96186" containerID="be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf" exitCode=0
Jul 29 01:26:37.907756 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.907687    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd" event={"ID":"7410469a-6dfe-471a-862f-322b49f96186","Type":"ContainerDied","Data":"be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf"}
Jul 29 01:26:37.913292 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.907926    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd" event={"ID":"7410469a-6dfe-471a-862f-322b49f96186","Type":"ContainerDied","Data":"b530a396e8edc93cc9abd46578921f8e54a323ade1ee206233ec7b81531a394f"}
Jul 29 01:26:37.913292 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.908003    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"
Jul 29 01:26:37.924207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.924191    2555 scope.go:117] "RemoveContainer" containerID="bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f"
Jul 29 01:26:37.924492 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:26:37.924473    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f\": container with ID starting with bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f not found: ID does not exist" containerID="bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f"
Jul 29 01:26:37.924547 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.924500    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f"} err="failed to get container status \"bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f\": rpc error: code = NotFound desc = could not find container \"bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f\": container with ID starting with bd30bd2434ec241384779ff120981fd8f6874c30832667be2997af6baf65763f not found: ID does not exist"
Jul 29 01:26:37.924547 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.924516    2555 scope.go:117] "RemoveContainer" containerID="f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43"
Jul 29 01:26:37.932289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.932257    2555 scope.go:117] "RemoveContainer" containerID="f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43"
Jul 29 01:26:37.932555 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:26:37.932532    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43\": container with ID starting with f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43 not found: ID does not exist" containerID="f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43"
Jul 29 01:26:37.932614 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.932564    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43"} err="failed to get container status \"f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43\": rpc error: code = NotFound desc = could not find container \"f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43\": container with ID starting with f4d2d4079fdda554bcfca42edfb303c136a3b497a8525637c4979eb3f6096f43 not found: ID does not exist"
Jul 29 01:26:37.932614 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.932586    2555 scope.go:117] "RemoveContainer" containerID="be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf"
Jul 29 01:26:37.940028 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.940013    2555 scope.go:117] "RemoveContainer" containerID="be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf"
Jul 29 01:26:37.940272 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:26:37.940252    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf\": container with ID starting with be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf not found: ID does not exist" containerID="be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf"
Jul 29 01:26:37.940419 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.940397    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf"} err="failed to get container status \"be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf\": rpc error: code = NotFound desc = could not find container \"be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf\": container with ID starting with be581cbf017acb98b0e4ed3285813423324abd19731026a07b2ff8ffd2092caf not found: ID does not exist"
Jul 29 01:26:37.961234 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.961213    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"]
Jul 29 01:26:37.964561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.964540    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-aaa0c308-57cc9b7d68-77ghp"]
Jul 29 01:26:37.975264 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.975243    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:37.977438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.977419    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"]
Jul 29 01:26:37.980151 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.980132    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-aaa0c308-7dbb7f66b4-l7kkd"]
Jul 29 01:26:37.991517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.991499    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"]
Jul 29 01:26:37.995455 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:37.995436    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-aaa0c308-9578b9777-z2zxq"]
Jul 29 01:26:38.224820 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:38.224789    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:38.452328 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:38.452297    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:39.190673 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:39.190640    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7410469a-6dfe-471a-862f-322b49f96186" path="/var/lib/kubelet/pods/7410469a-6dfe-471a-862f-322b49f96186/volumes"
Jul 29 01:26:39.190999 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:39.190986    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75345eba-8039-408f-b25e-c4c64257ebbd" path="/var/lib/kubelet/pods/75345eba-8039-408f-b25e-c4c64257ebbd/volumes"
Jul 29 01:26:39.191352 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:39.191338    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ec2da9-0fcd-4c9c-88ee-356b662792c3" path="/var/lib/kubelet/pods/c0ec2da9-0fcd-4c9c-88ee-356b662792c3/volumes"
Jul 29 01:26:40.163415 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.163377    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:40.261745 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.261713    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"]
Jul 29 01:26:40.262487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262369    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ec2da9-0fcd-4c9c-88ee-356b662792c3" containerName="maas-api"
Jul 29 01:26:40.262487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262393    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ec2da9-0fcd-4c9c-88ee-356b662792c3" containerName="maas-api"
Jul 29 01:26:40.262487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262410    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75345eba-8039-408f-b25e-c4c64257ebbd" containerName="payload-processing"
Jul 29 01:26:40.262487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262417    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="75345eba-8039-408f-b25e-c4c64257ebbd" containerName="payload-processing"
Jul 29 01:26:40.262487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262440    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7410469a-6dfe-471a-862f-322b49f96186" containerName="payload-pre-processing"
Jul 29 01:26:40.262487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262450    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7410469a-6dfe-471a-862f-322b49f96186" containerName="payload-pre-processing"
Jul 29 01:26:40.262954 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262566    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ec2da9-0fcd-4c9c-88ee-356b662792c3" containerName="maas-api"
Jul 29 01:26:40.262954 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262579    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7410469a-6dfe-471a-862f-322b49f96186" containerName="payload-pre-processing"
Jul 29 01:26:40.262954 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.262588    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="75345eba-8039-408f-b25e-c4c64257ebbd" containerName="payload-processing"
Jul 29 01:26:40.263636 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.263613    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:40.267658 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.267635    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.270553 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.270530    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8c8da55e-serving-cert\""
Jul 29 01:26:40.277788 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.277764    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"]
Jul 29 01:26:40.344020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.343988    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:40.359310 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.359260    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cad58959-fd0c-41e9-b33d-75f1950bfb24-maas-api-tls\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.359424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.359331    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cad58959-fd0c-41e9-b33d-75f1950bfb24-openshift-service-ca\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.359424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.359409    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx7d4\" (UniqueName: \"kubernetes.io/projected/cad58959-fd0c-41e9-b33d-75f1950bfb24-kube-api-access-fx7d4\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.460356 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.460327    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx7d4\" (UniqueName: \"kubernetes.io/projected/cad58959-fd0c-41e9-b33d-75f1950bfb24-kube-api-access-fx7d4\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.460497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.460413    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cad58959-fd0c-41e9-b33d-75f1950bfb24-maas-api-tls\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.460497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.460452    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cad58959-fd0c-41e9-b33d-75f1950bfb24-openshift-service-ca\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.461122 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.461088    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cad58959-fd0c-41e9-b33d-75f1950bfb24-openshift-service-ca\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.462754 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.462731    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cad58959-fd0c-41e9-b33d-75f1950bfb24-maas-api-tls\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.471992 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.471967    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx7d4\" (UniqueName: \"kubernetes.io/projected/cad58959-fd0c-41e9-b33d-75f1950bfb24-kube-api-access-fx7d4\") pod \"maas-api-e2e-ait-8c8da55e-6645544467-24ll2\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.476171 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.476149    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:40.580131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.580100    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:40.728946 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.728922    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"]
Jul 29 01:26:40.731144 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:26:40.731113    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad58959_fd0c_41e9_b33d_75f1950bfb24.slice/crio-2da5dc9e2a7f30bc985d7afa1e58d22737f0572a6bf766f44c7178181653e672 WatchSource:0}: Error finding container 2da5dc9e2a7f30bc985d7afa1e58d22737f0572a6bf766f44c7178181653e672: Status 404 returned error can't find the container with id 2da5dc9e2a7f30bc985d7afa1e58d22737f0572a6bf766f44c7178181653e672
Jul 29 01:26:40.865562 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.865528    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:40.926948 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.926903    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2" event={"ID":"cad58959-fd0c-41e9-b33d-75f1950bfb24","Type":"ContainerStarted","Data":"2da5dc9e2a7f30bc985d7afa1e58d22737f0572a6bf766f44c7178181653e672"}
Jul 29 01:26:40.963031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.962996    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:26:40.978739 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.978656    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"]
Jul 29 01:26:40.983568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.983550    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:40.986236 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.986217    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8c8da55e\""
Jul 29 01:26:40.986365 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.986285    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8c8da55e-dockercfg-6t8ql\""
Jul 29 01:26:40.993004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:40.992963    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"]
Jul 29 01:26:41.002556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.002530    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"]
Jul 29 01:26:41.006496 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.006475    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.017888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.017864    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"]
Jul 29 01:26:41.067386 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.067352    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6znd\" (UniqueName: \"kubernetes.io/projected/fc0a295c-c207-4398-abb3-f7c68a16735b-kube-api-access-x6znd\") pod \"payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.067539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.067444    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0a295c-c207-4398-abb3-f7c68a16735b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.168022 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.167985    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0a295c-c207-4398-abb3-f7c68a16735b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.168545 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.168056    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmtcn\" (UniqueName: \"kubernetes.io/projected/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-kube-api-access-pmtcn\") pod \"payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") " pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.168545 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.168093    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6znd\" (UniqueName: \"kubernetes.io/projected/fc0a295c-c207-4398-abb3-f7c68a16735b-kube-api-access-x6znd\") pod \"payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.168545 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.168129    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") " pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.168897 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.168873    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0a295c-c207-4398-abb3-f7c68a16735b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.176976 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.176954    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6znd\" (UniqueName: \"kubernetes.io/projected/fc0a295c-c207-4398-abb3-f7c68a16735b-kube-api-access-x6znd\") pod \"payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.268649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.268626    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") " pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.268753 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.268690    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmtcn\" (UniqueName: \"kubernetes.io/projected/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-kube-api-access-pmtcn\") pod \"payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") " pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.269249 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.269230    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-plugins-config-volume\") pod \"payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") " pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.279373 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.279351    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmtcn\" (UniqueName: \"kubernetes.io/projected/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-kube-api-access-pmtcn\") pod \"payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") " pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.295515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.295493    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.318595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.318577    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.652545 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.652518    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"]
Jul 29 01:26:41.654670 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:26:41.654645    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0a295c_c207_4398_abb3_f7c68a16735b.slice/crio-1757e4762f1a752cfcd596ed64b801131dcecb89dbf4746305e1b7f352034d2e WatchSource:0}: Error finding container 1757e4762f1a752cfcd596ed64b801131dcecb89dbf4746305e1b7f352034d2e: Status 404 returned error can't find the container with id 1757e4762f1a752cfcd596ed64b801131dcecb89dbf4746305e1b7f352034d2e
Jul 29 01:26:41.683979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.683953    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"]
Jul 29 01:26:41.692415 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:26:41.692391    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4f09d4_f026_4ff5_ada8_720f1f31b2c4.slice/crio-62954f2f75143bccc074f18da4cd6b0eac6b61e37eb344421e86c1bc0de346b6 WatchSource:0}: Error finding container 62954f2f75143bccc074f18da4cd6b0eac6b61e37eb344421e86c1bc0de346b6: Status 404 returned error can't find the container with id 62954f2f75143bccc074f18da4cd6b0eac6b61e37eb344421e86c1bc0de346b6
Jul 29 01:26:41.945692 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.945656    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2" event={"ID":"cad58959-fd0c-41e9-b33d-75f1950bfb24","Type":"ContainerStarted","Data":"3ee8bf6da563291376f082c7b2e5d40ffa0af9718e682a2b6d72cabbe774f7f0"}
Jul 29 01:26:41.945942 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.945930    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:41.948420 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.948390    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc" event={"ID":"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4","Type":"ContainerStarted","Data":"e83564c4a65de4f2cad3fce667b9b950e623b61cee5f768edadcc05e027bf12c"}
Jul 29 01:26:41.948570 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.948448    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:41.948570 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.948474    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc" event={"ID":"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4","Type":"ContainerStarted","Data":"62954f2f75143bccc074f18da4cd6b0eac6b61e37eb344421e86c1bc0de346b6"}
Jul 29 01:26:41.950304 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.950256    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x" event={"ID":"fc0a295c-c207-4398-abb3-f7c68a16735b","Type":"ContainerStarted","Data":"a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584"}
Jul 29 01:26:41.950409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.950327    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x" event={"ID":"fc0a295c-c207-4398-abb3-f7c68a16735b","Type":"ContainerStarted","Data":"1757e4762f1a752cfcd596ed64b801131dcecb89dbf4746305e1b7f352034d2e"}
Jul 29 01:26:41.950409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.950355    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:26:41.962990 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.962948    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2" podStartSLOduration=1.461057332 podStartE2EDuration="1.962935498s" totalImagesPullingTime="501.878166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:41.961350742 +0000 UTC m=+1931.292000355" watchObservedRunningTime="2026-07-29 01:26:41.962935498 +0000 UTC m=+1931.293585103"
Jul 29 01:26:41.979252 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.979213    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x" podStartSLOduration=1.979201415 podStartE2EDuration="1.979201415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:41.977068028 +0000 UTC m=+1931.307717635" watchObservedRunningTime="2026-07-29 01:26:41.979201415 +0000 UTC m=+1931.309851021"
Jul 29 01:26:41.993770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:41.993712    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc" podStartSLOduration=1.993696761 podStartE2EDuration="1.993696761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:26:41.993372437 +0000 UTC m=+1931.324022043" watchObservedRunningTime="2026-07-29 01:26:41.993696761 +0000 UTC m=+1931.324346369"
Jul 29 01:26:47.962060 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:47.962026    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:26:52.958497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:52.958459    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:26:52.958919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:26:52.958603    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:27:10.923958 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.923923    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"]
Jul 29 01:27:10.924671 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.924622    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2" podUID="cad58959-fd0c-41e9-b33d-75f1950bfb24" containerName="maas-api" containerID="cri-o://3ee8bf6da563291376f082c7b2e5d40ffa0af9718e682a2b6d72cabbe774f7f0" gracePeriod=30
Jul 29 01:27:10.952595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.952565    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:10.979154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.979117    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"]
Jul 29 01:27:10.979423 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.979392    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc" podUID="fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" containerName="payload-processing" containerID="cri-o://e83564c4a65de4f2cad3fce667b9b950e623b61cee5f768edadcc05e027bf12c" gracePeriod=30
Jul 29 01:27:10.985172 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.985142    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"]
Jul 29 01:27:10.985426 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:10.985388    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x" podUID="fc0a295c-c207-4398-abb3-f7c68a16735b" containerName="payload-pre-processing" containerID="cri-o://a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584" gracePeriod=30
Jul 29 01:27:11.021357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.021327    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:11.092914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.092895    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:11.101205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.101028    2555 generic.go:356] "Generic (PLEG): container finished" podID="cad58959-fd0c-41e9-b33d-75f1950bfb24" containerID="3ee8bf6da563291376f082c7b2e5d40ffa0af9718e682a2b6d72cabbe774f7f0" exitCode=0
Jul 29 01:27:11.101205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.101140    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2" event={"ID":"cad58959-fd0c-41e9-b33d-75f1950bfb24","Type":"ContainerDied","Data":"3ee8bf6da563291376f082c7b2e5d40ffa0af9718e682a2b6d72cabbe774f7f0"}
Jul 29 01:27:11.103058 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.103036    2555 generic.go:356] "Generic (PLEG): container finished" podID="fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" containerID="e83564c4a65de4f2cad3fce667b9b950e623b61cee5f768edadcc05e027bf12c" exitCode=0
Jul 29 01:27:11.103188 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.103118    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc" event={"ID":"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4","Type":"ContainerDied","Data":"e83564c4a65de4f2cad3fce667b9b950e623b61cee5f768edadcc05e027bf12c"}
Jul 29 01:27:11.198647 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.198570    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:11.239947 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.239919    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:27:11.259842 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.259814    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:27:11.292821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.292796    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:27:11.343577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343550    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx7d4\" (UniqueName: \"kubernetes.io/projected/cad58959-fd0c-41e9-b33d-75f1950bfb24-kube-api-access-fx7d4\") pod \"cad58959-fd0c-41e9-b33d-75f1950bfb24\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") "
Jul 29 01:27:11.343749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343599    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cad58959-fd0c-41e9-b33d-75f1950bfb24-openshift-service-ca\") pod \"cad58959-fd0c-41e9-b33d-75f1950bfb24\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") "
Jul 29 01:27:11.343749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343619    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmtcn\" (UniqueName: \"kubernetes.io/projected/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-kube-api-access-pmtcn\") pod \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") "
Jul 29 01:27:11.343749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343638    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-plugins-config-volume\") pod \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\" (UID: \"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4\") "
Jul 29 01:27:11.343749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343694    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0a295c-c207-4398-abb3-f7c68a16735b-plugins-config-volume\") pod \"fc0a295c-c207-4398-abb3-f7c68a16735b\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") "
Jul 29 01:27:11.343749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343713    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6znd\" (UniqueName: \"kubernetes.io/projected/fc0a295c-c207-4398-abb3-f7c68a16735b-kube-api-access-x6znd\") pod \"fc0a295c-c207-4398-abb3-f7c68a16735b\" (UID: \"fc0a295c-c207-4398-abb3-f7c68a16735b\") "
Jul 29 01:27:11.343749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.343743    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cad58959-fd0c-41e9-b33d-75f1950bfb24-maas-api-tls\") pod \"cad58959-fd0c-41e9-b33d-75f1950bfb24\" (UID: \"cad58959-fd0c-41e9-b33d-75f1950bfb24\") "
Jul 29 01:27:11.344081 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.344009    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cad58959-fd0c-41e9-b33d-75f1950bfb24-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cad58959-fd0c-41e9-b33d-75f1950bfb24" (UID: "cad58959-fd0c-41e9-b33d-75f1950bfb24"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:27:11.344081 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.344009    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" (UID: "fc4f09d4-f026-4ff5-ada8-720f1f31b2c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:27:11.344178 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.344116    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0a295c-c207-4398-abb3-f7c68a16735b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc0a295c-c207-4398-abb3-f7c68a16735b" (UID: "fc0a295c-c207-4398-abb3-f7c68a16735b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:27:11.345952 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.345923    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-kube-api-access-pmtcn" (OuterVolumeSpecName: "kube-api-access-pmtcn") pod "fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" (UID: "fc4f09d4-f026-4ff5-ada8-720f1f31b2c4"). InnerVolumeSpecName "kube-api-access-pmtcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:27:11.345952 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.345944    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cad58959-fd0c-41e9-b33d-75f1950bfb24-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cad58959-fd0c-41e9-b33d-75f1950bfb24" (UID: "cad58959-fd0c-41e9-b33d-75f1950bfb24"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:27:11.346100 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.345997    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0a295c-c207-4398-abb3-f7c68a16735b-kube-api-access-x6znd" (OuterVolumeSpecName: "kube-api-access-x6znd") pod "fc0a295c-c207-4398-abb3-f7c68a16735b" (UID: "fc0a295c-c207-4398-abb3-f7c68a16735b"). InnerVolumeSpecName "kube-api-access-x6znd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:27:11.346210 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.346192    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad58959-fd0c-41e9-b33d-75f1950bfb24-kube-api-access-fx7d4" (OuterVolumeSpecName: "kube-api-access-fx7d4") pod "cad58959-fd0c-41e9-b33d-75f1950bfb24" (UID: "cad58959-fd0c-41e9-b33d-75f1950bfb24"). InnerVolumeSpecName "kube-api-access-fx7d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445414    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cad58959-fd0c-41e9-b33d-75f1950bfb24-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445439    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx7d4\" (UniqueName: \"kubernetes.io/projected/cad58959-fd0c-41e9-b33d-75f1950bfb24-kube-api-access-fx7d4\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445450    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cad58959-fd0c-41e9-b33d-75f1950bfb24-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445459    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmtcn\" (UniqueName: \"kubernetes.io/projected/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-kube-api-access-pmtcn\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445468    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445477    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0a295c-c207-4398-abb3-f7c68a16735b-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:11.445486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:11.445486    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6znd\" (UniqueName: \"kubernetes.io/projected/fc0a295c-c207-4398-abb3-f7c68a16735b-kube-api-access-x6znd\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:12.109332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.109296    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"
Jul 29 01:27:12.109810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.109297    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2" event={"ID":"cad58959-fd0c-41e9-b33d-75f1950bfb24","Type":"ContainerDied","Data":"2da5dc9e2a7f30bc985d7afa1e58d22737f0572a6bf766f44c7178181653e672"}
Jul 29 01:27:12.109810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.109447    2555 scope.go:117] "RemoveContainer" containerID="3ee8bf6da563291376f082c7b2e5d40ffa0af9718e682a2b6d72cabbe774f7f0"
Jul 29 01:27:12.110850 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.110689    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"
Jul 29 01:27:12.110850 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.110700    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc" event={"ID":"fc4f09d4-f026-4ff5-ada8-720f1f31b2c4","Type":"ContainerDied","Data":"62954f2f75143bccc074f18da4cd6b0eac6b61e37eb344421e86c1bc0de346b6"}
Jul 29 01:27:12.112131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.112105    2555 generic.go:356] "Generic (PLEG): container finished" podID="fc0a295c-c207-4398-abb3-f7c68a16735b" containerID="a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584" exitCode=0
Jul 29 01:27:12.112250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.112158    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"
Jul 29 01:27:12.112250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.112188    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x" event={"ID":"fc0a295c-c207-4398-abb3-f7c68a16735b","Type":"ContainerDied","Data":"a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584"}
Jul 29 01:27:12.112250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.112238    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x" event={"ID":"fc0a295c-c207-4398-abb3-f7c68a16735b","Type":"ContainerDied","Data":"1757e4762f1a752cfcd596ed64b801131dcecb89dbf4746305e1b7f352034d2e"}
Jul 29 01:27:12.119437 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.119415    2555 scope.go:117] "RemoveContainer" containerID="e83564c4a65de4f2cad3fce667b9b950e623b61cee5f768edadcc05e027bf12c"
Jul 29 01:27:12.129465 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.129444    2555 scope.go:117] "RemoveContainer" containerID="a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584"
Jul 29 01:27:12.134598 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.134580    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"]
Jul 29 01:27:12.137543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.137521    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8c8da55e-6645544467-24ll2"]
Jul 29 01:27:12.138063 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.138048    2555 scope.go:117] "RemoveContainer" containerID="a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584"
Jul 29 01:27:12.138429 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:27:12.138408    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584\": container with ID starting with a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584 not found: ID does not exist" containerID="a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584"
Jul 29 01:27:12.138482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.138434    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584"} err="failed to get container status \"a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584\": rpc error: code = NotFound desc = could not find container \"a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584\": container with ID starting with a7d200eaa79cb42d0240931d1d7ca07667e0c5eb391ba68d47bbfdba97cfb584 not found: ID does not exist"
Jul 29 01:27:12.148406 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.148387    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"]
Jul 29 01:27:12.152453 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.152431    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8c8da55e-76c5ffc896-xq74x"]
Jul 29 01:27:12.162847 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.162824    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"]
Jul 29 01:27:12.167085 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.167065    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8c8da55e-55588fd67d-c7swc"]
Jul 29 01:27:12.263883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.263771    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:12.507661 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.507632    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:12.704449 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.704417    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:12.989212 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:12.989137    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:13.108495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:13.108457    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:13.188905 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:13.188875    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cad58959-fd0c-41e9-b33d-75f1950bfb24" path="/var/lib/kubelet/pods/cad58959-fd0c-41e9-b33d-75f1950bfb24/volumes"
Jul 29 01:27:13.189297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:13.189232    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc0a295c-c207-4398-abb3-f7c68a16735b" path="/var/lib/kubelet/pods/fc0a295c-c207-4398-abb3-f7c68a16735b/volumes"
Jul 29 01:27:13.189629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:13.189613    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" path="/var/lib/kubelet/pods/fc4f09d4-f026-4ff5-ada8-720f1f31b2c4/volumes"
Jul 29 01:27:14.140701 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:14.140665    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:15.412233 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.412187    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:15.499868 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.499836    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"]
Jul 29 01:27:15.500322 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500256    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0a295c-c207-4398-abb3-f7c68a16735b" containerName="payload-pre-processing"
Jul 29 01:27:15.500322 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500287    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0a295c-c207-4398-abb3-f7c68a16735b" containerName="payload-pre-processing"
Jul 29 01:27:15.500322 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500303    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" containerName="payload-processing"
Jul 29 01:27:15.500322 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500311    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" containerName="payload-processing"
Jul 29 01:27:15.500322 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500323    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cad58959-fd0c-41e9-b33d-75f1950bfb24" containerName="maas-api"
Jul 29 01:27:15.500617 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500328    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cad58959-fd0c-41e9-b33d-75f1950bfb24" containerName="maas-api"
Jul 29 01:27:15.500617 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500407    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cad58959-fd0c-41e9-b33d-75f1950bfb24" containerName="maas-api"
Jul 29 01:27:15.500617 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500419    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0a295c-c207-4398-abb3-f7c68a16735b" containerName="payload-pre-processing"
Jul 29 01:27:15.500617 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.500426    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4f09d4-f026-4ff5-ada8-720f1f31b2c4" containerName="payload-processing"
Jul 29 01:27:15.501588 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.501563    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:15.505200 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.505182    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.508367 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.508345    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c28408bd-serving-cert\""
Jul 29 01:27:15.516642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.516485    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"]
Jul 29 01:27:15.584669 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.584634    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmjl8\" (UniqueName: \"kubernetes.io/projected/625ee58a-e909-4f81-a0b6-d24311e1952f-kube-api-access-vmjl8\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.584829 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.584698    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.584829 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.584782    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625ee58a-e909-4f81-a0b6-d24311e1952f-openshift-service-ca\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.596524 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.596497    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:15.685497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.685409    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.685497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.685456    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625ee58a-e909-4f81-a0b6-d24311e1952f-openshift-service-ca\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.685738 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.685505    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmjl8\" (UniqueName: \"kubernetes.io/projected/625ee58a-e909-4f81-a0b6-d24311e1952f-kube-api-access-vmjl8\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.685738 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:27:15.685596    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-serving-cert: secret "maas-api-e2e-derive-c28408bd-serving-cert" not found
Jul 29 01:27:15.685738 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:27:15.685681    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls podName:625ee58a-e909-4f81-a0b6-d24311e1952f nodeName:}" failed. No retries permitted until 2026-07-29 01:27:16.185658387 +0000 UTC m=+1965.516307995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls") pod "maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" (UID: "625ee58a-e909-4f81-a0b6-d24311e1952f") : secret "maas-api-e2e-derive-c28408bd-serving-cert" not found
Jul 29 01:27:15.686147 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.686127    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625ee58a-e909-4f81-a0b6-d24311e1952f-openshift-service-ca\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:15.699779 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:15.699744    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmjl8\" (UniqueName: \"kubernetes.io/projected/625ee58a-e909-4f81-a0b6-d24311e1952f-kube-api-access-vmjl8\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:16.052622 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.052586    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:16.189791 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.189758    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:16.192244 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.192214    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls\") pod \"maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:16.287623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.287586    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:16.374144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.374065    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:16.378205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.378179    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"]
Jul 29 01:27:16.381916 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.381899    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.384786 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.384765    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-c28408bd\""
Jul 29 01:27:16.384910 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.384866    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-c28408bd-dockercfg-7nxw4\""
Jul 29 01:27:16.392482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.392460    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"]
Jul 29 01:27:16.419836 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.419811    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"]
Jul 29 01:27:16.423537 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.423521    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.423793 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.423769    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:16.442152 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.437466    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"]
Jul 29 01:27:16.496297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.495479    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpr7\" (UniqueName: \"kubernetes.io/projected/93f52180-6557-4401-aba3-d14d442cc60c-kube-api-access-7kpr7\") pod \"payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.496297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.495527    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86hvn\" (UniqueName: \"kubernetes.io/projected/50f56749-cbda-4136-ad56-6a9f0ea1369b-kube-api-access-86hvn\") pod \"payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") " pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.496297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.495632    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f52180-6557-4401-aba3-d14d442cc60c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.496297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.495709    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f56749-cbda-4136-ad56-6a9f0ea1369b-plugins-config-volume\") pod \"payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") " pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.595246 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.595222    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"]
Jul 29 01:27:16.596155 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.596130    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f56749-cbda-4136-ad56-6a9f0ea1369b-plugins-config-volume\") pod \"payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") " pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.596305 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.596188    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpr7\" (UniqueName: \"kubernetes.io/projected/93f52180-6557-4401-aba3-d14d442cc60c-kube-api-access-7kpr7\") pod \"payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.596305 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.596215    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86hvn\" (UniqueName: \"kubernetes.io/projected/50f56749-cbda-4136-ad56-6a9f0ea1369b-kube-api-access-86hvn\") pod \"payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") " pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.596433 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.596318    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f52180-6557-4401-aba3-d14d442cc60c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.597008 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.596987    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f52180-6557-4401-aba3-d14d442cc60c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.597110 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.597011    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f56749-cbda-4136-ad56-6a9f0ea1369b-plugins-config-volume\") pod \"payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") " pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.598122 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:27:16.598101    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625ee58a_e909_4f81_a0b6_d24311e1952f.slice/crio-540403bcd17f665d7ff49ee173ab4a17d9638457ef0b285614acbdb4edfad0ac WatchSource:0}: Error finding container 540403bcd17f665d7ff49ee173ab4a17d9638457ef0b285614acbdb4edfad0ac: Status 404 returned error can't find the container with id 540403bcd17f665d7ff49ee173ab4a17d9638457ef0b285614acbdb4edfad0ac
Jul 29 01:27:16.604446 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.604421    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpr7\" (UniqueName: \"kubernetes.io/projected/93f52180-6557-4401-aba3-d14d442cc60c-kube-api-access-7kpr7\") pod \"payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.604662 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.604642    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86hvn\" (UniqueName: \"kubernetes.io/projected/50f56749-cbda-4136-ad56-6a9f0ea1369b-kube-api-access-86hvn\") pod \"payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") " pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.694111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.694084    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:16.740249 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.739960    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:16.882846 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:16.882816    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"]
Jul 29 01:27:16.884503 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:27:16.884477    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f56749_cbda_4136_ad56_6a9f0ea1369b.slice/crio-0a1110d9a025ac563ec773991fc30aa1b814fd94cbd8dd3e27dd775152167e02 WatchSource:0}: Error finding container 0a1110d9a025ac563ec773991fc30aa1b814fd94cbd8dd3e27dd775152167e02: Status 404 returned error can't find the container with id 0a1110d9a025ac563ec773991fc30aa1b814fd94cbd8dd3e27dd775152167e02
Jul 29 01:27:17.059521 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.059467    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"]
Jul 29 01:27:17.062155 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:27:17.062117    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f52180_6557_4401_aba3_d14d442cc60c.slice/crio-6787286bc022acfa58929e7ec54d7aa3227024f023fa374e8786751561038bd3 WatchSource:0}: Error finding container 6787286bc022acfa58929e7ec54d7aa3227024f023fa374e8786751561038bd3: Status 404 returned error can't find the container with id 6787286bc022acfa58929e7ec54d7aa3227024f023fa374e8786751561038bd3
Jul 29 01:27:17.153302 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.153243    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" event={"ID":"625ee58a-e909-4f81-a0b6-d24311e1952f","Type":"ContainerStarted","Data":"0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37"}
Jul 29 01:27:17.153490 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.153341    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" event={"ID":"625ee58a-e909-4f81-a0b6-d24311e1952f","Type":"ContainerStarted","Data":"540403bcd17f665d7ff49ee173ab4a17d9638457ef0b285614acbdb4edfad0ac"}
Jul 29 01:27:17.153490 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.153379    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:17.154989 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.154948    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm" event={"ID":"93f52180-6557-4401-aba3-d14d442cc60c","Type":"ContainerStarted","Data":"c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef"}
Jul 29 01:27:17.155121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.155001    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:17.155121 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.155019    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm" event={"ID":"93f52180-6557-4401-aba3-d14d442cc60c","Type":"ContainerStarted","Data":"6787286bc022acfa58929e7ec54d7aa3227024f023fa374e8786751561038bd3"}
Jul 29 01:27:17.156585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.156562    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd" event={"ID":"50f56749-cbda-4136-ad56-6a9f0ea1369b","Type":"ContainerStarted","Data":"cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2"}
Jul 29 01:27:17.156697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.156603    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd" event={"ID":"50f56749-cbda-4136-ad56-6a9f0ea1369b","Type":"ContainerStarted","Data":"0a1110d9a025ac563ec773991fc30aa1b814fd94cbd8dd3e27dd775152167e02"}
Jul 29 01:27:17.156697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.156684    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:17.170792 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.170734    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" podStartSLOduration=1.7653436569999998 podStartE2EDuration="2.170716646s" totalImagesPullingTime="405.372989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:27:17.169970022 +0000 UTC m=+1966.500619628" watchObservedRunningTime="2026-07-29 01:27:17.170716646 +0000 UTC m=+1966.501366253"
Jul 29 01:27:17.189017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.188971    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm" podStartSLOduration=1.188957039 podStartE2EDuration="1.188957039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:27:17.187999736 +0000 UTC m=+1966.518649343" watchObservedRunningTime="2026-07-29 01:27:17.188957039 +0000 UTC m=+1966.519606645"
Jul 29 01:27:17.206572 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:17.206488    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd" podStartSLOduration=1.206474332 podStartE2EDuration="1.206474332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:27:17.204421313 +0000 UTC m=+1966.535070923" watchObservedRunningTime="2026-07-29 01:27:17.206474332 +0000 UTC m=+1966.537123937"
Jul 29 01:27:23.168978 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:23.168902    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:28.165746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:28.165714    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:28.166214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:28.165799    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:45.786763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.786695    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"]
Jul 29 01:27:45.787407 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.787002    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" podUID="625ee58a-e909-4f81-a0b6-d24311e1952f" containerName="maas-api" containerID="cri-o://0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37" gracePeriod=30
Jul 29 01:27:45.820469 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.820447    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:45.851955 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.851922    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"]
Jul 29 01:27:45.852660 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.852182    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd" podUID="50f56749-cbda-4136-ad56-6a9f0ea1369b" containerName="payload-processing" containerID="cri-o://cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2" gracePeriod=30
Jul 29 01:27:45.862079 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.861806    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"]
Jul 29 01:27:45.862079 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.862043    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm" podUID="93f52180-6557-4401-aba3-d14d442cc60c" containerName="payload-pre-processing" containerID="cri-o://c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef" gracePeriod=30
Jul 29 01:27:45.900075 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.900049    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:45.976901 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:45.976874    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:46.047579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.047471    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:46.167267 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.166820    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:46.172865 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.172844    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:46.177471 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.177448    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:46.266560 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.266528    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:46.274564 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274546    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls\") pod \"625ee58a-e909-4f81-a0b6-d24311e1952f\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") "
Jul 29 01:27:46.274635 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274596    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kpr7\" (UniqueName: \"kubernetes.io/projected/93f52180-6557-4401-aba3-d14d442cc60c-kube-api-access-7kpr7\") pod \"93f52180-6557-4401-aba3-d14d442cc60c\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") "
Jul 29 01:27:46.274670 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274653    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f56749-cbda-4136-ad56-6a9f0ea1369b-plugins-config-volume\") pod \"50f56749-cbda-4136-ad56-6a9f0ea1369b\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") "
Jul 29 01:27:46.274703 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274683    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f52180-6557-4401-aba3-d14d442cc60c-plugins-config-volume\") pod \"93f52180-6557-4401-aba3-d14d442cc60c\" (UID: \"93f52180-6557-4401-aba3-d14d442cc60c\") "
Jul 29 01:27:46.274751 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274715    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625ee58a-e909-4f81-a0b6-d24311e1952f-openshift-service-ca\") pod \"625ee58a-e909-4f81-a0b6-d24311e1952f\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") "
Jul 29 01:27:46.274785 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274763    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmjl8\" (UniqueName: \"kubernetes.io/projected/625ee58a-e909-4f81-a0b6-d24311e1952f-kube-api-access-vmjl8\") pod \"625ee58a-e909-4f81-a0b6-d24311e1952f\" (UID: \"625ee58a-e909-4f81-a0b6-d24311e1952f\") "
Jul 29 01:27:46.274835 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274786    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86hvn\" (UniqueName: \"kubernetes.io/projected/50f56749-cbda-4136-ad56-6a9f0ea1369b-kube-api-access-86hvn\") pod \"50f56749-cbda-4136-ad56-6a9f0ea1369b\" (UID: \"50f56749-cbda-4136-ad56-6a9f0ea1369b\") "
Jul 29 01:27:46.275028 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.274999    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f56749-cbda-4136-ad56-6a9f0ea1369b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50f56749-cbda-4136-ad56-6a9f0ea1369b" (UID: "50f56749-cbda-4136-ad56-6a9f0ea1369b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:27:46.275157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.275052    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f52180-6557-4401-aba3-d14d442cc60c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93f52180-6557-4401-aba3-d14d442cc60c" (UID: "93f52180-6557-4401-aba3-d14d442cc60c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:27:46.275157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.275116    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/625ee58a-e909-4f81-a0b6-d24311e1952f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "625ee58a-e909-4f81-a0b6-d24311e1952f" (UID: "625ee58a-e909-4f81-a0b6-d24311e1952f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:27:46.275347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.275122    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50f56749-cbda-4136-ad56-6a9f0ea1369b-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.275430 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.275359    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93f52180-6557-4401-aba3-d14d442cc60c-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.277020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.276997    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "625ee58a-e909-4f81-a0b6-d24311e1952f" (UID: "625ee58a-e909-4f81-a0b6-d24311e1952f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:27:46.277102 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.277083    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f56749-cbda-4136-ad56-6a9f0ea1369b-kube-api-access-86hvn" (OuterVolumeSpecName: "kube-api-access-86hvn") pod "50f56749-cbda-4136-ad56-6a9f0ea1369b" (UID: "50f56749-cbda-4136-ad56-6a9f0ea1369b"). InnerVolumeSpecName "kube-api-access-86hvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:27:46.277221 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.277195    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625ee58a-e909-4f81-a0b6-d24311e1952f-kube-api-access-vmjl8" (OuterVolumeSpecName: "kube-api-access-vmjl8") pod "625ee58a-e909-4f81-a0b6-d24311e1952f" (UID: "625ee58a-e909-4f81-a0b6-d24311e1952f"). InnerVolumeSpecName "kube-api-access-vmjl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:27:46.277260 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.277230    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93f52180-6557-4401-aba3-d14d442cc60c-kube-api-access-7kpr7" (OuterVolumeSpecName: "kube-api-access-7kpr7") pod "93f52180-6557-4401-aba3-d14d442cc60c" (UID: "93f52180-6557-4401-aba3-d14d442cc60c"). InnerVolumeSpecName "kube-api-access-7kpr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:27:46.310827 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.310737    2555 generic.go:356] "Generic (PLEG): container finished" podID="625ee58a-e909-4f81-a0b6-d24311e1952f" containerID="0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37" exitCode=0
Jul 29 01:27:46.310827 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.310773    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"
Jul 29 01:27:46.311024 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.310825    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" event={"ID":"625ee58a-e909-4f81-a0b6-d24311e1952f","Type":"ContainerDied","Data":"0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37"}
Jul 29 01:27:46.311024 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.310874    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2" event={"ID":"625ee58a-e909-4f81-a0b6-d24311e1952f","Type":"ContainerDied","Data":"540403bcd17f665d7ff49ee173ab4a17d9638457ef0b285614acbdb4edfad0ac"}
Jul 29 01:27:46.311024 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.310900    2555 scope.go:117] "RemoveContainer" containerID="0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37"
Jul 29 01:27:46.312533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.312508    2555 generic.go:356] "Generic (PLEG): container finished" podID="93f52180-6557-4401-aba3-d14d442cc60c" containerID="c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef" exitCode=0
Jul 29 01:27:46.312651 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.312604    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm" event={"ID":"93f52180-6557-4401-aba3-d14d442cc60c","Type":"ContainerDied","Data":"c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef"}
Jul 29 01:27:46.312651 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.312647    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm" event={"ID":"93f52180-6557-4401-aba3-d14d442cc60c","Type":"ContainerDied","Data":"6787286bc022acfa58929e7ec54d7aa3227024f023fa374e8786751561038bd3"}
Jul 29 01:27:46.312774 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.312706    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"
Jul 29 01:27:46.314486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.314462    2555 generic.go:356] "Generic (PLEG): container finished" podID="50f56749-cbda-4136-ad56-6a9f0ea1369b" containerID="cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2" exitCode=0
Jul 29 01:27:46.314605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.314507    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"
Jul 29 01:27:46.314605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.314537    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd" event={"ID":"50f56749-cbda-4136-ad56-6a9f0ea1369b","Type":"ContainerDied","Data":"cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2"}
Jul 29 01:27:46.314605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.314577    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd" event={"ID":"50f56749-cbda-4136-ad56-6a9f0ea1369b","Type":"ContainerDied","Data":"0a1110d9a025ac563ec773991fc30aa1b814fd94cbd8dd3e27dd775152167e02"}
Jul 29 01:27:46.321504 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.321485    2555 scope.go:117] "RemoveContainer" containerID="0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37"
Jul 29 01:27:46.322207 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:27:46.321801    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37\": container with ID starting with 0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37 not found: ID does not exist" containerID="0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37"
Jul 29 01:27:46.322324 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.322224    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37"} err="failed to get container status \"0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37\": rpc error: code = NotFound desc = could not find container \"0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37\": container with ID starting with 0487f4995d5902998dc1ce156bf63c4c8abb4cb0623f082aeaa25fa6609cdb37 not found: ID does not exist"
Jul 29 01:27:46.322324 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.322249    2555 scope.go:117] "RemoveContainer" containerID="c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef"
Jul 29 01:27:46.331730 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.331711    2555 scope.go:117] "RemoveContainer" containerID="c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef"
Jul 29 01:27:46.331996 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:27:46.331977    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef\": container with ID starting with c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef not found: ID does not exist" containerID="c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef"
Jul 29 01:27:46.332067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.332002    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef"} err="failed to get container status \"c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef\": rpc error: code = NotFound desc = could not find container \"c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef\": container with ID starting with c0062644f7ab500124d33a913bac46439693754179f93392c9020a658d61adef not found: ID does not exist"
Jul 29 01:27:46.332067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.332019    2555 scope.go:117] "RemoveContainer" containerID="cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2"
Jul 29 01:27:46.341414 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.341397    2555 scope.go:117] "RemoveContainer" containerID="cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2"
Jul 29 01:27:46.341677 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:27:46.341657    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2\": container with ID starting with cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2 not found: ID does not exist" containerID="cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2"
Jul 29 01:27:46.341738 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.341684    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2"} err="failed to get container status \"cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2\": rpc error: code = NotFound desc = could not find container \"cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2\": container with ID starting with cdae7c904b080009de7fc26f7acf735da80bde62bcf48f1ba8955cf4e10538c2 not found: ID does not exist"
Jul 29 01:27:46.346608 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.346575    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"]
Jul 29 01:27:46.357607 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.357583    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c28408bd-5ffdf74564-df2f2"]
Jul 29 01:27:46.376159 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.376134    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/625ee58a-e909-4f81-a0b6-d24311e1952f-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.376303 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.376162    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmjl8\" (UniqueName: \"kubernetes.io/projected/625ee58a-e909-4f81-a0b6-d24311e1952f-kube-api-access-vmjl8\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.376303 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.376177    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86hvn\" (UniqueName: \"kubernetes.io/projected/50f56749-cbda-4136-ad56-6a9f0ea1369b-kube-api-access-86hvn\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.376303 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.376187    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/625ee58a-e909-4f81-a0b6-d24311e1952f-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.376303 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.376200    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kpr7\" (UniqueName: \"kubernetes.io/projected/93f52180-6557-4401-aba3-d14d442cc60c-kube-api-access-7kpr7\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:27:46.383667 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.383636    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"]
Jul 29 01:27:46.386773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.386752    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c28408bd-ffbfb6556-wnppm"]
Jul 29 01:27:46.401060 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.401036    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"]
Jul 29 01:27:46.407344 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.407324    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c28408bd-68fb5fb47d-zzdzd"]
Jul 29 01:27:46.424612 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.424590    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:46.864367 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.864322    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:46.962355 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:46.962312    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:47.065002 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:47.064924    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:47.190312 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:47.190269    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f56749-cbda-4136-ad56-6a9f0ea1369b" path="/var/lib/kubelet/pods/50f56749-cbda-4136-ad56-6a9f0ea1369b/volumes"
Jul 29 01:27:47.190678 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:47.190657    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="625ee58a-e909-4f81-a0b6-d24311e1952f" path="/var/lib/kubelet/pods/625ee58a-e909-4f81-a0b6-d24311e1952f/volumes"
Jul 29 01:27:47.190982 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:47.190970    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f52180-6557-4401-aba3-d14d442cc60c" path="/var/lib/kubelet/pods/93f52180-6557-4401-aba3-d14d442cc60c/volumes"
Jul 29 01:27:48.512099 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:48.512067    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:54.722945 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:54.722913    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:54.832360 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:54.832325    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:54.926307 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:54.926261    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:55.027097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:55.027028    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:27:55.420650 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:27:55.420570    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:21.347332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:21.347297    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:27.435154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:27.435116    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:27.544640 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:27.544600    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:27.684459 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:27.684417    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:43.546343 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:43.546303    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:49.289107 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:49.289069    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:49.345165 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:49.345099    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:49.432546 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:49.432513    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:49.526773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:49.526739    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:28:59.997916 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:28:59.997830    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:00.797660 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:00.797628    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:07.393039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:07.392995    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:07.499931 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:07.499888    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:07.589975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:07.589939    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:07.695299 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:07.695240    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:21.221850 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:21.221812    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:21.317294 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:21.317239    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:21.418954 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:21.418918    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:23.073286 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:23.073233    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:29.226487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:29.226453    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:29.327975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:29.327941    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:29.920083 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:29.920048    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:36.158292 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:36.158242    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:36.256040 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:36.256006    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:36.393386 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:36.393358    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:36.701145 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:36.701114    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:36.951376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:36.951300    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:39.642861 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.642827    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:39.729440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.729408    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"]
Jul 29 01:29:39.729971 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.729952    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625ee58a-e909-4f81-a0b6-d24311e1952f" containerName="maas-api"
Jul 29 01:29:39.729971 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.729972    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="625ee58a-e909-4f81-a0b6-d24311e1952f" containerName="maas-api"
Jul 29 01:29:39.730126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.729983    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93f52180-6557-4401-aba3-d14d442cc60c" containerName="payload-pre-processing"
Jul 29 01:29:39.730126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.729993    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="93f52180-6557-4401-aba3-d14d442cc60c" containerName="payload-pre-processing"
Jul 29 01:29:39.730126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.730008    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f56749-cbda-4136-ad56-6a9f0ea1369b" containerName="payload-processing"
Jul 29 01:29:39.730126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.730018    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f56749-cbda-4136-ad56-6a9f0ea1369b" containerName="payload-processing"
Jul 29 01:29:39.730126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.730116    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="625ee58a-e909-4f81-a0b6-d24311e1952f" containerName="maas-api"
Jul 29 01:29:39.730126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.730128    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="93f52180-6557-4401-aba3-d14d442cc60c" containerName="payload-pre-processing"
Jul 29 01:29:39.730440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.730140    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f56749-cbda-4136-ad56-6a9f0ea1369b" containerName="payload-processing"
Jul 29 01:29:39.733895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.733869    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.737069 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.736879    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4bfdf293-serving-cert\""
Jul 29 01:29:39.737930 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.737896    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:39.748795 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.748706    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"]
Jul 29 01:29:39.838589 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.838547    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:39.868789 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.868746    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpfxl\" (UniqueName: \"kubernetes.io/projected/2e156e7a-4863-405a-b891-92a076ce6872-kube-api-access-wpfxl\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.868984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.868834    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.868984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.868865    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e156e7a-4863-405a-b891-92a076ce6872-openshift-service-ca\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.969743 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.969709    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.969919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.969750    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e156e7a-4863-405a-b891-92a076ce6872-openshift-service-ca\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.969919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.969830    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpfxl\" (UniqueName: \"kubernetes.io/projected/2e156e7a-4863-405a-b891-92a076ce6872-kube-api-access-wpfxl\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.969919 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:29:39.969860    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-serving-cert: secret "maas-api-e2e-mt-4bfdf293-serving-cert" not found
Jul 29 01:29:39.970031 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:29:39.969976    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls podName:2e156e7a-4863-405a-b891-92a076ce6872 nodeName:}" failed. No retries permitted until 2026-07-29 01:29:40.46995323 +0000 UTC m=+2109.800602815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls") pod "maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" (UID: "2e156e7a-4863-405a-b891-92a076ce6872") : secret "maas-api-e2e-mt-4bfdf293-serving-cert" not found
Jul 29 01:29:39.970485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.970465    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e156e7a-4863-405a-b891-92a076ce6872-openshift-service-ca\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:39.985075 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:39.985045    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpfxl\" (UniqueName: \"kubernetes.io/projected/2e156e7a-4863-405a-b891-92a076ce6872-kube-api-access-wpfxl\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:40.043209 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.043175    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:40.470864 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.470831    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:40.474857 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.474828    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:40.477452 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.477427    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls\") pod \"maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:40.575186 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.575145    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:29:40.648752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.648716    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:40.666572 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.666542    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"]
Jul 29 01:29:40.670626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.670604    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.673380 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.673352    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4bfdf293-dockercfg-pmkcw\""
Jul 29 01:29:40.673516 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.673351    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4bfdf293\""
Jul 29 01:29:40.680072 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.680030    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"]
Jul 29 01:29:40.762575 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.762544    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"]
Jul 29 01:29:40.767586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.767561    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:40.777753 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.777619    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmg5b\" (UniqueName: \"kubernetes.io/projected/043745d3-50da-47d8-9c70-230060e94d9a-kube-api-access-rmg5b\") pod \"payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.779606 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.777878    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/043745d3-50da-47d8-9c70-230060e94d9a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.779606 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.779484    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"]
Jul 29 01:29:40.813392 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.813350    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"]
Jul 29 01:29:40.815813 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:29:40.815786    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e156e7a_4863_405a_b891_92a076ce6872.slice/crio-6903d303e6542019401b9a545907ffdbbf839be94296e16c87dca019f7ac4d27 WatchSource:0}: Error finding container 6903d303e6542019401b9a545907ffdbbf839be94296e16c87dca019f7ac4d27: Status 404 returned error can't find the container with id 6903d303e6542019401b9a545907ffdbbf839be94296e16c87dca019f7ac4d27
Jul 29 01:29:40.880135 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.880097    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/043745d3-50da-47d8-9c70-230060e94d9a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.880347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.880149    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l424p\" (UniqueName: \"kubernetes.io/projected/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-kube-api-access-l424p\") pod \"payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") " pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:40.880347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.880188    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmg5b\" (UniqueName: \"kubernetes.io/projected/043745d3-50da-47d8-9c70-230060e94d9a-kube-api-access-rmg5b\") pod \"payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.880347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.880206    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-plugins-config-volume\") pod \"payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") " pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:40.880824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.880805    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/043745d3-50da-47d8-9c70-230060e94d9a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.888434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.888410    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmg5b\" (UniqueName: \"kubernetes.io/projected/043745d3-50da-47d8-9c70-230060e94d9a-kube-api-access-rmg5b\") pod \"payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.897860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.897833    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" event={"ID":"2e156e7a-4863-405a-b891-92a076ce6872","Type":"ContainerStarted","Data":"6903d303e6542019401b9a545907ffdbbf839be94296e16c87dca019f7ac4d27"}
Jul 29 01:29:40.981816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.981777    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l424p\" (UniqueName: \"kubernetes.io/projected/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-kube-api-access-l424p\") pod \"payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") " pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:40.982008 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.981881    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-plugins-config-volume\") pod \"payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") " pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:40.982637 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.982610    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-plugins-config-volume\") pod \"payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") " pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:40.987630 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.987604    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:40.992568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:40.992540    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l424p\" (UniqueName: \"kubernetes.io/projected/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-kube-api-access-l424p\") pod \"payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") " pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:41.089840 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.089807    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:41.330597 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.330570    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"]
Jul 29 01:29:41.333208 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:29:41.333180    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043745d3_50da_47d8_9c70_230060e94d9a.slice/crio-b5c1dc860ad340e4296c4a016dc07fa7016cc7a65d1a89b2b489b3ddc314e247 WatchSource:0}: Error finding container b5c1dc860ad340e4296c4a016dc07fa7016cc7a65d1a89b2b489b3ddc314e247: Status 404 returned error can't find the container with id b5c1dc860ad340e4296c4a016dc07fa7016cc7a65d1a89b2b489b3ddc314e247
Jul 29 01:29:41.439243 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.439214    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"]
Jul 29 01:29:41.441890 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:29:41.441857    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88fc7fb2_12f6_4d4f_a3f7_3637bd9af83f.slice/crio-3bb3dfab61a56e81708e4c8cb629924eafae969e14071c52b4995f8db8720550 WatchSource:0}: Error finding container 3bb3dfab61a56e81708e4c8cb629924eafae969e14071c52b4995f8db8720550: Status 404 returned error can't find the container with id 3bb3dfab61a56e81708e4c8cb629924eafae969e14071c52b4995f8db8720550
Jul 29 01:29:41.904342 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.904303    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn" event={"ID":"043745d3-50da-47d8-9c70-230060e94d9a","Type":"ContainerStarted","Data":"e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9"}
Jul 29 01:29:41.904816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.904355    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn" event={"ID":"043745d3-50da-47d8-9c70-230060e94d9a","Type":"ContainerStarted","Data":"b5c1dc860ad340e4296c4a016dc07fa7016cc7a65d1a89b2b489b3ddc314e247"}
Jul 29 01:29:41.904816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.904378    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:29:41.905863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.905841    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm" event={"ID":"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f","Type":"ContainerStarted","Data":"17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639"}
Jul 29 01:29:41.905985 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.905883    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm" event={"ID":"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f","Type":"ContainerStarted","Data":"3bb3dfab61a56e81708e4c8cb629924eafae969e14071c52b4995f8db8720550"}
Jul 29 01:29:41.905985 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.905908    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:41.907264 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.907240    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" event={"ID":"2e156e7a-4863-405a-b891-92a076ce6872","Type":"ContainerStarted","Data":"5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097"}
Jul 29 01:29:41.907401 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.907322    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:41.922757 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.922714    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn" podStartSLOduration=1.922700979 podStartE2EDuration="1.922700979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:29:41.920682403 +0000 UTC m=+2111.251332012" watchObservedRunningTime="2026-07-29 01:29:41.922700979 +0000 UTC m=+2111.253350585"
Jul 29 01:29:41.939447 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.939400    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm" podStartSLOduration=1.93938657 podStartE2EDuration="1.93938657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:29:41.937862939 +0000 UTC m=+2111.268512547" watchObservedRunningTime="2026-07-29 01:29:41.93938657 +0000 UTC m=+2111.270036177"
Jul 29 01:29:41.956944 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:41.956895    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" podStartSLOduration=2.509335134 podStartE2EDuration="2.956879599s" totalImagesPullingTime="447.544465ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:29:41.955836488 +0000 UTC m=+2111.286486098" watchObservedRunningTime="2026-07-29 01:29:41.956879599 +0000 UTC m=+2111.287529204"
Jul 29 01:29:47.917771 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:47.917741    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:29:52.914495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:52.914460    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:29:52.914971 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:29:52.914746    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:30:04.584338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:04.584299    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:05.240709 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:05.240671    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:05.340197 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:05.340162    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:05.660636 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:05.660554    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:11.469039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:11.469006    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:11.539721 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:11.539690    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:22.347132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.347097    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"]
Jul 29 01:30:22.347663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.347393    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" podUID="2e156e7a-4863-405a-b891-92a076ce6872" containerName="maas-api" containerID="cri-o://5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097" gracePeriod=30
Jul 29 01:30:22.370047 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.370014    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:22.403165 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.403134    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"]
Jul 29 01:30:22.403424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.403397    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm" podUID="88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" containerName="payload-processing" containerID="cri-o://17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639" gracePeriod=30
Jul 29 01:30:22.412197 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.411834    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"]
Jul 29 01:30:22.412197 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.412039    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn" podUID="043745d3-50da-47d8-9c70-230060e94d9a" containerName="payload-pre-processing" containerID="cri-o://e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9" gracePeriod=30
Jul 29 01:30:22.446821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.446795    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:22.516932 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.516893    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:22.619577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.619494    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:22.688981 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.688961    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:30:22.695980 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.695952    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:30:22.715545 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.715522    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:30:22.748307 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.748235    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmg5b\" (UniqueName: \"kubernetes.io/projected/043745d3-50da-47d8-9c70-230060e94d9a-kube-api-access-rmg5b\") pod \"043745d3-50da-47d8-9c70-230060e94d9a\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") "
Jul 29 01:30:22.748503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.748314    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpfxl\" (UniqueName: \"kubernetes.io/projected/2e156e7a-4863-405a-b891-92a076ce6872-kube-api-access-wpfxl\") pod \"2e156e7a-4863-405a-b891-92a076ce6872\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") "
Jul 29 01:30:22.748828 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.748678    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l424p\" (UniqueName: \"kubernetes.io/projected/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-kube-api-access-l424p\") pod \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") "
Jul 29 01:30:22.749298 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.749254    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/043745d3-50da-47d8-9c70-230060e94d9a-plugins-config-volume\") pod \"043745d3-50da-47d8-9c70-230060e94d9a\" (UID: \"043745d3-50da-47d8-9c70-230060e94d9a\") "
Jul 29 01:30:22.749477 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.749457    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls\") pod \"2e156e7a-4863-405a-b891-92a076ce6872\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") "
Jul 29 01:30:22.749632 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.749616    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-plugins-config-volume\") pod \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\" (UID: \"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f\") "
Jul 29 01:30:22.749824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.749808    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e156e7a-4863-405a-b891-92a076ce6872-openshift-service-ca\") pod \"2e156e7a-4863-405a-b891-92a076ce6872\" (UID: \"2e156e7a-4863-405a-b891-92a076ce6872\") "
Jul 29 01:30:22.750547 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.750522    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e156e7a-4863-405a-b891-92a076ce6872-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e156e7a-4863-405a-b891-92a076ce6872" (UID: "2e156e7a-4863-405a-b891-92a076ce6872"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:30:22.751070 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.751048    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/043745d3-50da-47d8-9c70-230060e94d9a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "043745d3-50da-47d8-9c70-230060e94d9a" (UID: "043745d3-50da-47d8-9c70-230060e94d9a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:30:22.751234 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.751206    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e156e7a-4863-405a-b891-92a076ce6872-kube-api-access-wpfxl" (OuterVolumeSpecName: "kube-api-access-wpfxl") pod "2e156e7a-4863-405a-b891-92a076ce6872" (UID: "2e156e7a-4863-405a-b891-92a076ce6872"). InnerVolumeSpecName "kube-api-access-wpfxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:30:22.751801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.751776    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" (UID: "88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:30:22.752783 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.752753    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/043745d3-50da-47d8-9c70-230060e94d9a-kube-api-access-rmg5b" (OuterVolumeSpecName: "kube-api-access-rmg5b") pod "043745d3-50da-47d8-9c70-230060e94d9a" (UID: "043745d3-50da-47d8-9c70-230060e94d9a"). InnerVolumeSpecName "kube-api-access-rmg5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:30:22.754389 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.754357    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e156e7a-4863-405a-b891-92a076ce6872" (UID: "2e156e7a-4863-405a-b891-92a076ce6872"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:30:22.754486 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.754377    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-kube-api-access-l424p" (OuterVolumeSpecName: "kube-api-access-l424p") pod "88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" (UID: "88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f"). InnerVolumeSpecName "kube-api-access-l424p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:30:22.850703 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850667    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e156e7a-4863-405a-b891-92a076ce6872-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.850703 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850694    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmg5b\" (UniqueName: \"kubernetes.io/projected/043745d3-50da-47d8-9c70-230060e94d9a-kube-api-access-rmg5b\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.850703 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850705    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpfxl\" (UniqueName: \"kubernetes.io/projected/2e156e7a-4863-405a-b891-92a076ce6872-kube-api-access-wpfxl\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.850939 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850714    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l424p\" (UniqueName: \"kubernetes.io/projected/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-kube-api-access-l424p\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.850939 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850724    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/043745d3-50da-47d8-9c70-230060e94d9a-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.850939 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850733    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e156e7a-4863-405a-b891-92a076ce6872-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.850939 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.850741    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:30:22.999032 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:22.998999    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:23.117665 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.117632    2555 generic.go:356] "Generic (PLEG): container finished" podID="2e156e7a-4863-405a-b891-92a076ce6872" containerID="5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097" exitCode=0
Jul 29 01:30:23.117849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.117693    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"
Jul 29 01:30:23.117849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.117726    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" event={"ID":"2e156e7a-4863-405a-b891-92a076ce6872","Type":"ContainerDied","Data":"5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097"}
Jul 29 01:30:23.117849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.117809    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x" event={"ID":"2e156e7a-4863-405a-b891-92a076ce6872","Type":"ContainerDied","Data":"6903d303e6542019401b9a545907ffdbbf839be94296e16c87dca019f7ac4d27"}
Jul 29 01:30:23.117849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.117829    2555 scope.go:117] "RemoveContainer" containerID="5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097"
Jul 29 01:30:23.119088 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.119066    2555 generic.go:356] "Generic (PLEG): container finished" podID="043745d3-50da-47d8-9c70-230060e94d9a" containerID="e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9" exitCode=0
Jul 29 01:30:23.119189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.119114    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"
Jul 29 01:30:23.119189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.119150    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn" event={"ID":"043745d3-50da-47d8-9c70-230060e94d9a","Type":"ContainerDied","Data":"e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9"}
Jul 29 01:30:23.119320 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.119203    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn" event={"ID":"043745d3-50da-47d8-9c70-230060e94d9a","Type":"ContainerDied","Data":"b5c1dc860ad340e4296c4a016dc07fa7016cc7a65d1a89b2b489b3ddc314e247"}
Jul 29 01:30:23.120543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.120522    2555 generic.go:356] "Generic (PLEG): container finished" podID="88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" containerID="17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639" exitCode=0
Jul 29 01:30:23.120643 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.120564    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm" event={"ID":"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f","Type":"ContainerDied","Data":"17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639"}
Jul 29 01:30:23.120643 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.120580    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"
Jul 29 01:30:23.120643 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.120605    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm" event={"ID":"88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f","Type":"ContainerDied","Data":"3bb3dfab61a56e81708e4c8cb629924eafae969e14071c52b4995f8db8720550"}
Jul 29 01:30:23.128120 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.128099    2555 scope.go:117] "RemoveContainer" containerID="5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097"
Jul 29 01:30:23.128432 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:30:23.128405    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097\": container with ID starting with 5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097 not found: ID does not exist" containerID="5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097"
Jul 29 01:30:23.128529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.128458    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097"} err="failed to get container status \"5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097\": rpc error: code = NotFound desc = could not find container \"5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097\": container with ID starting with 5c1c7f24963ea0eb2c77fba563270764db6f37fe329a63b9f46092db7fbc6097 not found: ID does not exist"
Jul 29 01:30:23.128529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.128480    2555 scope.go:117] "RemoveContainer" containerID="e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9"
Jul 29 01:30:23.138077 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.138060    2555 scope.go:117] "RemoveContainer" containerID="e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9"
Jul 29 01:30:23.138362 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:30:23.138339    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9\": container with ID starting with e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9 not found: ID does not exist" containerID="e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9"
Jul 29 01:30:23.138417 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.138373    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9"} err="failed to get container status \"e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9\": rpc error: code = NotFound desc = could not find container \"e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9\": container with ID starting with e6025a39cbdaef43e3bddafbc9eac17fb4eb70f6f70ef7555201b7bad9e0bfe9 not found: ID does not exist"
Jul 29 01:30:23.138417 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.138395    2555 scope.go:117] "RemoveContainer" containerID="17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639"
Jul 29 01:30:23.144027 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.144005    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"]
Jul 29 01:30:23.147104 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.147085    2555 scope.go:117] "RemoveContainer" containerID="17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639"
Jul 29 01:30:23.147398 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:30:23.147374    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639\": container with ID starting with 17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639 not found: ID does not exist" containerID="17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639"
Jul 29 01:30:23.147458 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.147407    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639"} err="failed to get container status \"17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639\": rpc error: code = NotFound desc = could not find container \"17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639\": container with ID starting with 17162e9646fb32ff8f41c9a99cef779052f75a2e171b10ca662a7abd676bf639 not found: ID does not exist"
Jul 29 01:30:23.148065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.148047    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4bfdf293-55c7cd4749-f946x"]
Jul 29 01:30:23.158772 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.158752    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"]
Jul 29 01:30:23.162956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.162935    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4bfdf293-b5c767fb-nf8nn"]
Jul 29 01:30:23.175426 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.175400    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"]
Jul 29 01:30:23.182651 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.182631    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4bfdf293-7677bf46bb-pvxcm"]
Jul 29 01:30:23.191059 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.191033    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="043745d3-50da-47d8-9c70-230060e94d9a" path="/var/lib/kubelet/pods/043745d3-50da-47d8-9c70-230060e94d9a/volumes"
Jul 29 01:30:23.191463 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.191449    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e156e7a-4863-405a-b891-92a076ce6872" path="/var/lib/kubelet/pods/2e156e7a-4863-405a-b891-92a076ce6872/volumes"
Jul 29 01:30:23.191780 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:23.191768    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" path="/var/lib/kubelet/pods/88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f/volumes"
Jul 29 01:30:28.748333 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:28.748300    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:28.848958 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:28.848910    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:29.013689 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:29.013603    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:29.218233 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:29.218202    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:29.339533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:29.339443    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:29.439902 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:29.439873    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:29.563291 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:29.563254    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:35.280941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:35.280906    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:35.382796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:35.382764    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:35.486792 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:35.486761    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:35.681095 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:35.681012    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:41.251700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:41.251669    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:41.320986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:41.320959    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:43.775768 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:43.775739    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:48.066463 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:48.066430    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:48.355344 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:48.355252    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:48.618425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:48.618340    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.106917 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.106874    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.187820 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.187786    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.195067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195041    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"]
Jul 29 01:30:51.195432 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195419    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="043745d3-50da-47d8-9c70-230060e94d9a" containerName="payload-pre-processing"
Jul 29 01:30:51.195482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195433    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="043745d3-50da-47d8-9c70-230060e94d9a" containerName="payload-pre-processing"
Jul 29 01:30:51.195482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195451    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" containerName="payload-processing"
Jul 29 01:30:51.195482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195457    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" containerName="payload-processing"
Jul 29 01:30:51.195482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195466    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e156e7a-4863-405a-b891-92a076ce6872" containerName="maas-api"
Jul 29 01:30:51.195482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195471    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e156e7a-4863-405a-b891-92a076ce6872" containerName="maas-api"
Jul 29 01:30:51.195649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195529    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e156e7a-4863-405a-b891-92a076ce6872" containerName="maas-api"
Jul 29 01:30:51.195649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195537    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="043745d3-50da-47d8-9c70-230060e94d9a" containerName="payload-pre-processing"
Jul 29 01:30:51.195649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.195545    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="88fc7fb2-12f6-4d4f-a3f7-3637bd9af83f" containerName="payload-processing"
Jul 29 01:30:51.198588 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.198572    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.201165 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.201146    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f46b2363-serving-cert\""
Jul 29 01:30:51.210914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.210892    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"]
Jul 29 01:30:51.250935 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.250907    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.322084 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.322040    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-openshift-service-ca\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.322332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.322131    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-maas-api-tls\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.322332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.322155    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n77gp\" (UniqueName: \"kubernetes.io/projected/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-kube-api-access-n77gp\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.423539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.423440    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-maas-api-tls\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.423539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.423489    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n77gp\" (UniqueName: \"kubernetes.io/projected/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-kube-api-access-n77gp\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.423793 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.423579    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-openshift-service-ca\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.424377 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.424346    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-openshift-service-ca\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.424502 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.424351    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.425975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.425957    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-maas-api-tls\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.432472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.432445    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n77gp\" (UniqueName: \"kubernetes.io/projected/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-kube-api-access-n77gp\") pod \"maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.508224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.508190    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:51.651321 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.651296    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"]
Jul 29 01:30:51.653066 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:30:51.653030    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e51ea55_2840_4eed_95e2_4ab7a694ebc3.slice/crio-62b88737890c96b4a0a5e8b04982d3eaf9c00f871b768d437b7eb87ca98b97db WatchSource:0}: Error finding container 62b88737890c96b4a0a5e8b04982d3eaf9c00f871b768d437b7eb87ca98b97db: Status 404 returned error can't find the container with id 62b88737890c96b4a0a5e8b04982d3eaf9c00f871b768d437b7eb87ca98b97db
Jul 29 01:30:51.654580 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.654559    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:30:51.796743 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.796711    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.890806 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.890772    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:30:51.898560 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.898514    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"]
Jul 29 01:30:51.903144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.903124    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:51.905886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.905866    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f46b2363-dockercfg-lq6gs\""
Jul 29 01:30:51.905997 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.905898    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f46b2363\""
Jul 29 01:30:51.918303 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.917346    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"]
Jul 29 01:30:51.927677 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.927652    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:51.927781 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.927686    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tskcs\" (UniqueName: \"kubernetes.io/projected/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-kube-api-access-tskcs\") pod \"payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:51.937694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.937667    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"]
Jul 29 01:30:51.941476 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.941457    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:51.950797 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:51.950774    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"]
Jul 29 01:30:52.028539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.028505    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-plugins-config-volume\") pod \"payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") " pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.028719 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.028647    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:52.028719 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.028681    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtvv4\" (UniqueName: \"kubernetes.io/projected/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-kube-api-access-dtvv4\") pod \"payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") " pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.028719 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.028706    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tskcs\" (UniqueName: \"kubernetes.io/projected/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-kube-api-access-tskcs\") pod \"payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:52.029591 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.029562    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:52.037532 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.037500    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tskcs\" (UniqueName: \"kubernetes.io/projected/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-kube-api-access-tskcs\") pod \"payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:52.129261 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.129196    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-plugins-config-volume\") pod \"payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") " pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.129690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.129334    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtvv4\" (UniqueName: \"kubernetes.io/projected/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-kube-api-access-dtvv4\") pod \"payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") " pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.130024 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.129999    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-plugins-config-volume\") pod \"payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") " pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.137760 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.137724    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtvv4\" (UniqueName: \"kubernetes.io/projected/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-kube-api-access-dtvv4\") pod \"payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") " pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.216237 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.216205    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:52.253680 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.253639    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:52.268314 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.268218    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff" event={"ID":"6e51ea55-2840-4eed-95e2-4ab7a694ebc3","Type":"ContainerStarted","Data":"62b88737890c96b4a0a5e8b04982d3eaf9c00f871b768d437b7eb87ca98b97db"}
Jul 29 01:30:52.368210 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.368166    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"]
Jul 29 01:30:52.370808 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:30:52.370784    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbaf46d_a2c9_449e_9aed_8a7ea430149f.slice/crio-6d83641d32286c3983295a30aef9182f28ca6aec6e87a053dbab1d190b52657f WatchSource:0}: Error finding container 6d83641d32286c3983295a30aef9182f28ca6aec6e87a053dbab1d190b52657f: Status 404 returned error can't find the container with id 6d83641d32286c3983295a30aef9182f28ca6aec6e87a053dbab1d190b52657f
Jul 29 01:30:52.415561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:52.415538    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"]
Jul 29 01:30:52.417395 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:30:52.417368    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c87d518_a05e_4fb7_8d3c_c7a4085a7242.slice/crio-a62212d021a5e67a18bac1829d92d011f3968c5c21da469f2821673a596bb8db WatchSource:0}: Error finding container a62212d021a5e67a18bac1829d92d011f3968c5c21da469f2821673a596bb8db: Status 404 returned error can't find the container with id a62212d021a5e67a18bac1829d92d011f3968c5c21da469f2821673a596bb8db
Jul 29 01:30:53.275494 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.275304    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs" event={"ID":"8fbaf46d-a2c9-449e-9aed-8a7ea430149f","Type":"ContainerStarted","Data":"122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5"}
Jul 29 01:30:53.275494 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.275375    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs" event={"ID":"8fbaf46d-a2c9-449e-9aed-8a7ea430149f","Type":"ContainerStarted","Data":"6d83641d32286c3983295a30aef9182f28ca6aec6e87a053dbab1d190b52657f"}
Jul 29 01:30:53.276022 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.275524    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:30:53.277022 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.276993    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff" event={"ID":"6e51ea55-2840-4eed-95e2-4ab7a694ebc3","Type":"ContainerStarted","Data":"e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d"}
Jul 29 01:30:53.277179 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.277090    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:30:53.278642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.278605    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br" event={"ID":"7c87d518-a05e-4fb7-8d3c-c7a4085a7242","Type":"ContainerStarted","Data":"dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145"}
Jul 29 01:30:53.278752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.278651    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br" event={"ID":"7c87d518-a05e-4fb7-8d3c-c7a4085a7242","Type":"ContainerStarted","Data":"a62212d021a5e67a18bac1829d92d011f3968c5c21da469f2821673a596bb8db"}
Jul 29 01:30:53.278752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.278681    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:30:53.293302 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.293223    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs" podStartSLOduration=2.293209615 podStartE2EDuration="2.293209615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:30:53.291850854 +0000 UTC m=+2182.622500460" watchObservedRunningTime="2026-07-29 01:30:53.293209615 +0000 UTC m=+2182.623859220"
Jul 29 01:30:53.309294 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.309233    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff" podStartSLOduration=1.679281512 podStartE2EDuration="2.309219366s" totalImagesPullingTime="629.937854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:30:53.307459868 +0000 UTC m=+2182.638109467" watchObservedRunningTime="2026-07-29 01:30:53.309219366 +0000 UTC m=+2182.639868972"
Jul 29 01:30:53.324907 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:53.324853    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br" podStartSLOduration=2.324835678 podStartE2EDuration="2.324835678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:30:53.323545553 +0000 UTC m=+2182.654195156" watchObservedRunningTime="2026-07-29 01:30:53.324835678 +0000 UTC m=+2182.655485337"
Jul 29 01:30:59.289814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:30:59.289785    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:31:04.286367 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:31:04.286332    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:31:04.288771 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:31:04.286404    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:31:10.317394 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:31:10.317359    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:31:15.939291 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:31:15.939250    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:31:20.603214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:31:20.603179    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:10.854255 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.853729    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"]
Jul 29 01:32:10.854255 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.853953    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff" podUID="6e51ea55-2840-4eed-95e2-4ab7a694ebc3" containerName="maas-api" containerID="cri-o://e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d" gracePeriod=30
Jul 29 01:32:10.875980 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.875952    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:10.913032 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.912991    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"]
Jul 29 01:32:10.913320 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.913289    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br" podUID="7c87d518-a05e-4fb7-8d3c-c7a4085a7242" containerName="payload-processing" containerID="cri-o://dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145" gracePeriod=30
Jul 29 01:32:10.920268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.919974    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"]
Jul 29 01:32:10.920268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.920181    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs" podUID="8fbaf46d-a2c9-449e-9aed-8a7ea430149f" containerName="payload-pre-processing" containerID="cri-o://122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5" gracePeriod=30
Jul 29 01:32:10.952998 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:10.952962    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:11.026657 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.026619    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:11.103295 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.103171    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:11.214793 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.214768    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:32:11.221058 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.221035    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:32:11.240728 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.240702    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:32:11.348896 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.348862    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n77gp\" (UniqueName: \"kubernetes.io/projected/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-kube-api-access-n77gp\") pod \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") "
Jul 29 01:32:11.348896 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.348898    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtvv4\" (UniqueName: \"kubernetes.io/projected/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-kube-api-access-dtvv4\") pod \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") "
Jul 29 01:32:11.349111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.348931    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-openshift-service-ca\") pod \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") "
Jul 29 01:32:11.349111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.348954    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-plugins-config-volume\") pod \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") "
Jul 29 01:32:11.349227 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349151    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tskcs\" (UniqueName: \"kubernetes.io/projected/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-kube-api-access-tskcs\") pod \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\" (UID: \"8fbaf46d-a2c9-449e-9aed-8a7ea430149f\") "
Jul 29 01:32:11.349227 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349193    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-plugins-config-volume\") pod \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\" (UID: \"7c87d518-a05e-4fb7-8d3c-c7a4085a7242\") "
Jul 29 01:32:11.349363 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349311    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8fbaf46d-a2c9-449e-9aed-8a7ea430149f" (UID: "8fbaf46d-a2c9-449e-9aed-8a7ea430149f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:32:11.349363 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349319    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-maas-api-tls\") pod \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\" (UID: \"6e51ea55-2840-4eed-95e2-4ab7a694ebc3\") "
Jul 29 01:32:11.349363 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349334    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e51ea55-2840-4eed-95e2-4ab7a694ebc3" (UID: "6e51ea55-2840-4eed-95e2-4ab7a694ebc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:32:11.349702 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349670    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c87d518-a05e-4fb7-8d3c-c7a4085a7242" (UID: "7c87d518-a05e-4fb7-8d3c-c7a4085a7242"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:32:11.349819 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349801    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.349880 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349821    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.349880 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.349836    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.351070 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.351044    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-kube-api-access-dtvv4" (OuterVolumeSpecName: "kube-api-access-dtvv4") pod "7c87d518-a05e-4fb7-8d3c-c7a4085a7242" (UID: "7c87d518-a05e-4fb7-8d3c-c7a4085a7242"). InnerVolumeSpecName "kube-api-access-dtvv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:32:11.351070 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.351057    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-kube-api-access-n77gp" (OuterVolumeSpecName: "kube-api-access-n77gp") pod "6e51ea55-2840-4eed-95e2-4ab7a694ebc3" (UID: "6e51ea55-2840-4eed-95e2-4ab7a694ebc3"). InnerVolumeSpecName "kube-api-access-n77gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:32:11.351229 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.351195    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e51ea55-2840-4eed-95e2-4ab7a694ebc3" (UID: "6e51ea55-2840-4eed-95e2-4ab7a694ebc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:32:11.351271 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.351220    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-kube-api-access-tskcs" (OuterVolumeSpecName: "kube-api-access-tskcs") pod "8fbaf46d-a2c9-449e-9aed-8a7ea430149f" (UID: "8fbaf46d-a2c9-449e-9aed-8a7ea430149f"). InnerVolumeSpecName "kube-api-access-tskcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:32:11.450579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.450543    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tskcs\" (UniqueName: \"kubernetes.io/projected/8fbaf46d-a2c9-449e-9aed-8a7ea430149f-kube-api-access-tskcs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.450579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.450571    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.450579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.450581    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n77gp\" (UniqueName: \"kubernetes.io/projected/6e51ea55-2840-4eed-95e2-4ab7a694ebc3-kube-api-access-n77gp\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.450868 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.450593    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtvv4\" (UniqueName: \"kubernetes.io/projected/7c87d518-a05e-4fb7-8d3c-c7a4085a7242-kube-api-access-dtvv4\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:32:11.669549 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.669511    2555 generic.go:356] "Generic (PLEG): container finished" podID="8fbaf46d-a2c9-449e-9aed-8a7ea430149f" containerID="122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5" exitCode=0
Jul 29 01:32:11.669729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.669568    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"
Jul 29 01:32:11.669729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.669601    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs" event={"ID":"8fbaf46d-a2c9-449e-9aed-8a7ea430149f","Type":"ContainerDied","Data":"122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5"}
Jul 29 01:32:11.669729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.669692    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs" event={"ID":"8fbaf46d-a2c9-449e-9aed-8a7ea430149f","Type":"ContainerDied","Data":"6d83641d32286c3983295a30aef9182f28ca6aec6e87a053dbab1d190b52657f"}
Jul 29 01:32:11.669729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.669722    2555 scope.go:117] "RemoveContainer" containerID="122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5"
Jul 29 01:32:11.670942 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.670916    2555 generic.go:356] "Generic (PLEG): container finished" podID="6e51ea55-2840-4eed-95e2-4ab7a694ebc3" containerID="e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d" exitCode=0
Jul 29 01:32:11.671041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.670945    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff" event={"ID":"6e51ea55-2840-4eed-95e2-4ab7a694ebc3","Type":"ContainerDied","Data":"e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d"}
Jul 29 01:32:11.671041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.670980    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"
Jul 29 01:32:11.671041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.670990    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff" event={"ID":"6e51ea55-2840-4eed-95e2-4ab7a694ebc3","Type":"ContainerDied","Data":"62b88737890c96b4a0a5e8b04982d3eaf9c00f871b768d437b7eb87ca98b97db"}
Jul 29 01:32:11.672442 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.672418    2555 generic.go:356] "Generic (PLEG): container finished" podID="7c87d518-a05e-4fb7-8d3c-c7a4085a7242" containerID="dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145" exitCode=0
Jul 29 01:32:11.672556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.672455    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br" event={"ID":"7c87d518-a05e-4fb7-8d3c-c7a4085a7242","Type":"ContainerDied","Data":"dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145"}
Jul 29 01:32:11.672556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.672500    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br" event={"ID":"7c87d518-a05e-4fb7-8d3c-c7a4085a7242","Type":"ContainerDied","Data":"a62212d021a5e67a18bac1829d92d011f3968c5c21da469f2821673a596bb8db"}
Jul 29 01:32:11.672556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.672511    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"
Jul 29 01:32:11.680240 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.680204    2555 scope.go:117] "RemoveContainer" containerID="122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5"
Jul 29 01:32:11.680544 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:32:11.680506    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5\": container with ID starting with 122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5 not found: ID does not exist" containerID="122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5"
Jul 29 01:32:11.680648 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.680620    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5"} err="failed to get container status \"122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5\": rpc error: code = NotFound desc = could not find container \"122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5\": container with ID starting with 122500921e65ec5fd4c3b7ac1c55e0b54ebbb4e72d0db80e9eda22462b1650d5 not found: ID does not exist"
Jul 29 01:32:11.680730 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.680652    2555 scope.go:117] "RemoveContainer" containerID="e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d"
Jul 29 01:32:11.690097 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.690080    2555 scope.go:117] "RemoveContainer" containerID="e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d"
Jul 29 01:32:11.690380 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:32:11.690354    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d\": container with ID starting with e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d not found: ID does not exist" containerID="e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d"
Jul 29 01:32:11.690473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.690391    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d"} err="failed to get container status \"e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d\": rpc error: code = NotFound desc = could not find container \"e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d\": container with ID starting with e2faa9f36a0d011d6f454c1398c74851a6499f5375e5d2cc06dbf12a28dca67d not found: ID does not exist"
Jul 29 01:32:11.690473 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.690410    2555 scope.go:117] "RemoveContainer" containerID="dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145"
Jul 29 01:32:11.698963 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.698947    2555 scope.go:117] "RemoveContainer" containerID="dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145"
Jul 29 01:32:11.699137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.699119    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"]
Jul 29 01:32:11.699209 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:32:11.699174    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145\": container with ID starting with dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145 not found: ID does not exist" containerID="dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145"
Jul 29 01:32:11.699261 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.699199    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145"} err="failed to get container status \"dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145\": rpc error: code = NotFound desc = could not find container \"dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145\": container with ID starting with dd3577be99754f642017719fa1725e993eec4141616fb21bf423db0c87e56145 not found: ID does not exist"
Jul 29 01:32:11.704899 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.704852    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f46b2363-788ffdd4d4-q88bs"]
Jul 29 01:32:11.716178 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.716159    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"]
Jul 29 01:32:11.721913 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.721893    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f46b2363-75ff6b5d75-wgtff"]
Jul 29 01:32:11.736143 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.736123    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"]
Jul 29 01:32:11.740732 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:11.740709    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f46b2363-5c8888f5fd-546br"]
Jul 29 01:32:13.189747 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:13.189713    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e51ea55-2840-4eed-95e2-4ab7a694ebc3" path="/var/lib/kubelet/pods/6e51ea55-2840-4eed-95e2-4ab7a694ebc3/volumes"
Jul 29 01:32:13.190156 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:13.190062    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c87d518-a05e-4fb7-8d3c-c7a4085a7242" path="/var/lib/kubelet/pods/7c87d518-a05e-4fb7-8d3c-c7a4085a7242/volumes"
Jul 29 01:32:13.190399 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:13.190385    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fbaf46d-a2c9-449e-9aed-8a7ea430149f" path="/var/lib/kubelet/pods/8fbaf46d-a2c9-449e-9aed-8a7ea430149f/volumes"
Jul 29 01:32:14.181596 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:14.181550    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:19.212457 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:19.212423    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:55.299932 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:55.299896    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:32:56.223782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:32:56.223747    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:01.687877 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:01.687841    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:01.753296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:01.753243    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:01.966619 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:01.966582    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:02.174930 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:02.174896    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:03.222268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:03.222229    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:03.315709 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:03.315677    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:03.415899 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:03.415867    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:04.105972 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:04.105937    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:04.206587 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:04.206555    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:04.308681 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:04.308652    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:05.613883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:05.613851    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:11.268191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:11.268158    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:11.325779 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:11.325748    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:11.926702 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:11.926662    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:12.255874 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:12.255838    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:12.356642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:12.356608    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:17.782574 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:17.782538    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:17.892270 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:17.892217    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:17.982839 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:17.982789    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:28.457360 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:28.457240    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:28.810093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:28.809971    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:48.988597 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:48.988561    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:50.382046 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:50.382002    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:55.981956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:55.981909    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:56.082756 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:56.082719    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:56.320453 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:56.320354    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:56.622041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:56.621952    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:56.686533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:56.686483    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:33:59.939009 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:33:59.938971    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:00.035865 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.035813    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:00.043008 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.042979    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"]
Jul 29 01:34:00.043642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.043600    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c87d518-a05e-4fb7-8d3c-c7a4085a7242" containerName="payload-processing"
Jul 29 01:34:00.043642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.043632    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c87d518-a05e-4fb7-8d3c-c7a4085a7242" containerName="payload-processing"
Jul 29 01:34:00.043918 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.043654    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e51ea55-2840-4eed-95e2-4ab7a694ebc3" containerName="maas-api"
Jul 29 01:34:00.043918 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.043663    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e51ea55-2840-4eed-95e2-4ab7a694ebc3" containerName="maas-api"
Jul 29 01:34:00.043918 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.043770    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fbaf46d-a2c9-449e-9aed-8a7ea430149f" containerName="payload-pre-processing"
Jul 29 01:34:00.043918 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.043783    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbaf46d-a2c9-449e-9aed-8a7ea430149f" containerName="payload-pre-processing"
Jul 29 01:34:00.044126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.044020    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e51ea55-2840-4eed-95e2-4ab7a694ebc3" containerName="maas-api"
Jul 29 01:34:00.044126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.044037    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c87d518-a05e-4fb7-8d3c-c7a4085a7242" containerName="payload-processing"
Jul 29 01:34:00.044126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.044053    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fbaf46d-a2c9-449e-9aed-8a7ea430149f" containerName="payload-pre-processing"
Jul 29 01:34:00.047581 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.047560    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.050755 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.050736    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6afea2-serving-cert\""
Jul 29 01:34:00.059783 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.059757    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"]
Jul 29 01:34:00.120315 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.120262    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/096be4f7-6eb2-421a-bdeb-82022493eba3-maas-api-tls\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.120460 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.120353    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f946\" (UniqueName: \"kubernetes.io/projected/096be4f7-6eb2-421a-bdeb-82022493eba3-kube-api-access-2f946\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.120460 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.120429    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/096be4f7-6eb2-421a-bdeb-82022493eba3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.136883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.136849    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:00.221542 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.221459    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/096be4f7-6eb2-421a-bdeb-82022493eba3-maas-api-tls\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.221542 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.221513    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f946\" (UniqueName: \"kubernetes.io/projected/096be4f7-6eb2-421a-bdeb-82022493eba3-kube-api-access-2f946\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.221773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.221583    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/096be4f7-6eb2-421a-bdeb-82022493eba3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.222257 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.222230    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/096be4f7-6eb2-421a-bdeb-82022493eba3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.223942 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.223921    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/096be4f7-6eb2-421a-bdeb-82022493eba3-maas-api-tls\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.237792 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.237765    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f946\" (UniqueName: \"kubernetes.io/projected/096be4f7-6eb2-421a-bdeb-82022493eba3-kube-api-access-2f946\") pod \"maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.241763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.241730    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:00.360640 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.360606    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:00.500777 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.500754    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"]
Jul 29 01:34:00.502798 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:34:00.502769    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096be4f7_6eb2_421a_bdeb_82022493eba3.slice/crio-86dff26d126e129b8116e29b06a37b17d50d1af1e600e4353a6d91a9b8798559 WatchSource:0}: Error finding container 86dff26d126e129b8116e29b06a37b17d50d1af1e600e4353a6d91a9b8798559: Status 404 returned error can't find the container with id 86dff26d126e129b8116e29b06a37b17d50d1af1e600e4353a6d91a9b8798559
Jul 29 01:34:00.661648 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.661613    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:00.754700 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.754622    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:00.757289 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.757257    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"]
Jul 29 01:34:00.761986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.761969    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:00.764750 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.764728    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6afea2\""
Jul 29 01:34:00.771970 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.771950    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"]
Jul 29 01:34:00.802232 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.802194    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"]
Jul 29 01:34:00.808376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.808349    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:00.811768 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.811742    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6afea2-dockercfg-xnq88\""
Jul 29 01:34:00.811915 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.811808    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"]
Jul 29 01:34:00.827151 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.827122    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/573aa670-a87e-4b53-b7f5-cdedb9312306-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:00.827266 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.827172    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqx4p\" (UniqueName: \"kubernetes.io/projected/573aa670-a87e-4b53-b7f5-cdedb9312306-kube-api-access-hqx4p\") pod \"payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:00.928498 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.928383    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/573aa670-a87e-4b53-b7f5-cdedb9312306-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:00.928498 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.928424    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqx4p\" (UniqueName: \"kubernetes.io/projected/573aa670-a87e-4b53-b7f5-cdedb9312306-kube-api-access-hqx4p\") pod \"payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:00.928498 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.928486    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:00.928685 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.928517    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9np\" (UniqueName: \"kubernetes.io/projected/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-kube-api-access-cc9np\") pod \"payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:00.929375 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.929341    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/573aa670-a87e-4b53-b7f5-cdedb9312306-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:00.938122 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:00.938083    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqx4p\" (UniqueName: \"kubernetes.io/projected/573aa670-a87e-4b53-b7f5-cdedb9312306-kube-api-access-hqx4p\") pod \"payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:01.033570 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.030099    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:01.033570 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.031062    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9np\" (UniqueName: \"kubernetes.io/projected/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-kube-api-access-cc9np\") pod \"payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:01.033570 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.031682    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:01.038513 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.038493    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9np\" (UniqueName: \"kubernetes.io/projected/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-kube-api-access-cc9np\") pod \"payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:01.073940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.073915    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:01.120816 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.120787    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:01.213189 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.213132    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc" event={"ID":"096be4f7-6eb2-421a-bdeb-82022493eba3","Type":"ContainerStarted","Data":"ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1"}
Jul 29 01:34:01.213425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.213199    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc" event={"ID":"096be4f7-6eb2-421a-bdeb-82022493eba3","Type":"ContainerStarted","Data":"86dff26d126e129b8116e29b06a37b17d50d1af1e600e4353a6d91a9b8798559"}
Jul 29 01:34:01.213425 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.213264    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:01.224953 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.224929    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"]
Jul 29 01:34:01.233228 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.233178    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc" podStartSLOduration=0.808329497 podStartE2EDuration="1.233158135s" totalImagesPullingTime="424.828638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:01.230328826 +0000 UTC m=+2370.560978430" watchObservedRunningTime="2026-07-29 01:34:01.233158135 +0000 UTC m=+2370.563807742"
Jul 29 01:34:01.237594 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:34:01.237564    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573aa670_a87e_4b53_b7f5_cdedb9312306.slice/crio-7b92e8509b8f007eaf2fab1fc005812b33796108e80d0b2eeae13534c1b9adb9 WatchSource:0}: Error finding container 7b92e8509b8f007eaf2fab1fc005812b33796108e80d0b2eeae13534c1b9adb9: Status 404 returned error can't find the container with id 7b92e8509b8f007eaf2fab1fc005812b33796108e80d0b2eeae13534c1b9adb9
Jul 29 01:34:01.288033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:01.287929    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"]
Jul 29 01:34:01.297428 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:34:01.297402    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb129efc_ebc4_4bee_a8cb_6a5d3316598b.slice/crio-5a43748af73373667e1f5b33f4c98a1f13726d5ddd201567bae1a21b74145867 WatchSource:0}: Error finding container 5a43748af73373667e1f5b33f4c98a1f13726d5ddd201567bae1a21b74145867: Status 404 returned error can't find the container with id 5a43748af73373667e1f5b33f4c98a1f13726d5ddd201567bae1a21b74145867
Jul 29 01:34:02.218791 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.218756    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw" event={"ID":"fb129efc-ebc4-4bee-a8cb-6a5d3316598b","Type":"ContainerStarted","Data":"794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced"}
Jul 29 01:34:02.219256 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.218808    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw" event={"ID":"fb129efc-ebc4-4bee-a8cb-6a5d3316598b","Type":"ContainerStarted","Data":"5a43748af73373667e1f5b33f4c98a1f13726d5ddd201567bae1a21b74145867"}
Jul 29 01:34:02.219256 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.218864    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:02.220292 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.220255    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg" event={"ID":"573aa670-a87e-4b53-b7f5-cdedb9312306","Type":"ContainerStarted","Data":"f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff"}
Jul 29 01:34:02.220465 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.220451    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:02.220537 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.220483    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg" event={"ID":"573aa670-a87e-4b53-b7f5-cdedb9312306","Type":"ContainerStarted","Data":"7b92e8509b8f007eaf2fab1fc005812b33796108e80d0b2eeae13534c1b9adb9"}
Jul 29 01:34:02.237945 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.237882    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw" podStartSLOduration=2.237863723 podStartE2EDuration="2.237863723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:02.236241782 +0000 UTC m=+2371.566891393" watchObservedRunningTime="2026-07-29 01:34:02.237863723 +0000 UTC m=+2371.568513331"
Jul 29 01:34:02.252706 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:02.252662    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg" podStartSLOduration=2.252643571 podStartE2EDuration="2.252643571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:02.250568055 +0000 UTC m=+2371.581217663" watchObservedRunningTime="2026-07-29 01:34:02.252643571 +0000 UTC m=+2371.583293426"
Jul 29 01:34:07.225949 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:07.225921    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:34:13.228163 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:13.228125    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:34:13.228576 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:13.228293    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:34:13.795363 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:13.795329    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:24.901802 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:24.901763    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:25.236971 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:25.236939    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:25.580603 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:25.580508    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:25.676208 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:25.676176    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:28.189045 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.189002    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:28.286295 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.286252    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:28.417522 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.417490    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"]
Jul 29 01:34:28.421413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.421392    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.424127 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.424107    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-f5ec11-serving-cert\""
Jul 29 01:34:28.431927 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.431902    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"]
Jul 29 01:34:28.459768 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.459694    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:28.488805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.488776    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.488929 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.488818    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx9xt\" (UniqueName: \"kubernetes.io/projected/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-kube-api-access-mx9xt\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.488929 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.488860    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.590228 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.590195    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx9xt\" (UniqueName: \"kubernetes.io/projected/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-kube-api-access-mx9xt\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.590433 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.590258    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.590433 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.590381    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.590531 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:34:28.590441    2555 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-serving-cert: secret "maas-api-e2e-inf-b-f5ec11-serving-cert" not found
Jul 29 01:34:28.590531 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:34:28.590514    2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls podName:a8f0ccb8-5199-45a8-ae25-99bf183f1a93 nodeName:}" failed. No retries permitted until 2026-07-29 01:34:29.090499194 +0000 UTC m=+2398.421148779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls") pod "maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" (UID: "a8f0ccb8-5199-45a8-ae25-99bf183f1a93") : secret "maas-api-e2e-inf-b-f5ec11-serving-cert" not found
Jul 29 01:34:28.591052 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.591034    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.601780 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.601752    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx9xt\" (UniqueName: \"kubernetes.io/projected/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-kube-api-access-mx9xt\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:28.764447 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:28.764336    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:29.054751 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.054668    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:29.094839 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.094809    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:29.097154 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.097127    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls\") pod \"maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:29.157407 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.157377    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:29.175582 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.175551    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"]
Jul 29 01:34:29.179048 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.179031    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.182007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.181986    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-f5ec11-dockercfg-7hs5l\""
Jul 29 01:34:29.182124 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.182050    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-f5ec11\""
Jul 29 01:34:29.192075 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.192049    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"]
Jul 29 01:34:29.197715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.197681    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr8k8\" (UniqueName: \"kubernetes.io/projected/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-kube-api-access-rr8k8\") pod \"payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.197831 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.197784    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.207518 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.207483    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"]
Jul 29 01:34:29.211934 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.211913    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.218108 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.218079    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"]
Jul 29 01:34:29.298699 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.298665    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr8k8\" (UniqueName: \"kubernetes.io/projected/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-kube-api-access-rr8k8\") pod \"payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.298851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.298716    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf5wz\" (UniqueName: \"kubernetes.io/projected/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-kube-api-access-pf5wz\") pod \"payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.298851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.298789    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.298991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.298901    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.299485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.299468    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.308017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.307961    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr8k8\" (UniqueName: \"kubernetes.io/projected/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-kube-api-access-rr8k8\") pod \"payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.333381 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.333339    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:29.399624 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.399592    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.399773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.399705    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf5wz\" (UniqueName: \"kubernetes.io/projected/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-kube-api-access-pf5wz\") pod \"payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.400373 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.400349    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.413226 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.413199    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf5wz\" (UniqueName: \"kubernetes.io/projected/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-kube-api-access-pf5wz\") pod \"payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.474739 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.474713    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"]
Jul 29 01:34:29.476892 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:34:29.476867    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f0ccb8_5199_45a8_ae25_99bf183f1a93.slice/crio-882f5cac6348d61800b38c5fafc33ac8d5698bfd60ec7d86ed69d93181d10128 WatchSource:0}: Error finding container 882f5cac6348d61800b38c5fafc33ac8d5698bfd60ec7d86ed69d93181d10128: Status 404 returned error can't find the container with id 882f5cac6348d61800b38c5fafc33ac8d5698bfd60ec7d86ed69d93181d10128
Jul 29 01:34:29.491774 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.491755    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:29.525432 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.525407    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:29.683858 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.683825    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"]
Jul 29 01:34:29.685765 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:34:29.685738    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7959ec2f_edc7_4f17_9ec1_72dbb6708a0e.slice/crio-894d69d86ce3a6ebbb1c5dd75e357594db0f0e79b93cbcb5f13f415646bafc7d WatchSource:0}: Error finding container 894d69d86ce3a6ebbb1c5dd75e357594db0f0e79b93cbcb5f13f415646bafc7d: Status 404 returned error can't find the container with id 894d69d86ce3a6ebbb1c5dd75e357594db0f0e79b93cbcb5f13f415646bafc7d
Jul 29 01:34:29.849509 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:29.849246    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"]
Jul 29 01:34:29.852142 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:34:29.852099    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ba9cb5_bce3_4bc1_b7df_5520d56b3cf6.slice/crio-9c4a9a566b48dfbe397036ab92dcd4aa7c6477513bc1c66f36e619e1317bc994 WatchSource:0}: Error finding container 9c4a9a566b48dfbe397036ab92dcd4aa7c6477513bc1c66f36e619e1317bc994: Status 404 returned error can't find the container with id 9c4a9a566b48dfbe397036ab92dcd4aa7c6477513bc1c66f36e619e1317bc994
Jul 29 01:34:30.369217 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.369181    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" event={"ID":"a8f0ccb8-5199-45a8-ae25-99bf183f1a93","Type":"ContainerStarted","Data":"ab9c70de3a7866220a4eecefa65ee617c43e52e7c579b6db9194a502bba0f32f"}
Jul 29 01:34:30.369759 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.369249    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:30.369759 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.369297    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" event={"ID":"a8f0ccb8-5199-45a8-ae25-99bf183f1a93","Type":"ContainerStarted","Data":"882f5cac6348d61800b38c5fafc33ac8d5698bfd60ec7d86ed69d93181d10128"}
Jul 29 01:34:30.370894 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.370865    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp" event={"ID":"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6","Type":"ContainerStarted","Data":"8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463"}
Jul 29 01:34:30.371018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.370918    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:30.371018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.370940    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp" event={"ID":"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6","Type":"ContainerStarted","Data":"9c4a9a566b48dfbe397036ab92dcd4aa7c6477513bc1c66f36e619e1317bc994"}
Jul 29 01:34:30.372451 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.372429    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj" event={"ID":"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e","Type":"ContainerStarted","Data":"6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3"}
Jul 29 01:34:30.372605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.372471    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj" event={"ID":"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e","Type":"ContainerStarted","Data":"894d69d86ce3a6ebbb1c5dd75e357594db0f0e79b93cbcb5f13f415646bafc7d"}
Jul 29 01:34:30.372605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.372498    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:30.392834 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.392782    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" podStartSLOduration=2.039174523 podStartE2EDuration="2.392767159s" totalImagesPullingTime="353.592636ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:30.390463916 +0000 UTC m=+2399.721113521" watchObservedRunningTime="2026-07-29 01:34:30.392767159 +0000 UTC m=+2399.723416765"
Jul 29 01:34:30.411018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.410961    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj" podStartSLOduration=1.410947351 podStartE2EDuration="1.410947351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:30.409522847 +0000 UTC m=+2399.740172454" watchObservedRunningTime="2026-07-29 01:34:30.410947351 +0000 UTC m=+2399.741596957"
Jul 29 01:34:30.426127 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:30.426075    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp" podStartSLOduration=1.426059741 podStartE2EDuration="1.426059741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:34:30.424850353 +0000 UTC m=+2399.755499983" watchObservedRunningTime="2026-07-29 01:34:30.426059741 +0000 UTC m=+2399.756709347"
Jul 29 01:34:36.383543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:36.383505    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:34:41.379245 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:41.379209    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:34:41.379688 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:41.379305    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:34:53.074560 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:53.074472    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:34:57.729430 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:34:57.729386    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:35:23.358813 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:35:23.358781    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:35:44.888318 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:35:44.888260    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:07.553698 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:07.553655    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:08.611693 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:08.611663    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:19.282443 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.282398    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"]
Jul 29 01:36:19.282982 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.282650    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc" podUID="096be4f7-6eb2-421a-bdeb-82022493eba3" containerName="maas-api" containerID="cri-o://ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1" gracePeriod=30
Jul 29 01:36:19.310639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.310612    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:19.341774 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.341740    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"]
Jul 29 01:36:19.341986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.341957    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw" podUID="fb129efc-ebc4-4bee-a8cb-6a5d3316598b" containerName="payload-processing" containerID="cri-o://794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced" gracePeriod=30
Jul 29 01:36:19.351561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.351532    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"]
Jul 29 01:36:19.351825 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.351800    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg" podUID="573aa670-a87e-4b53-b7f5-cdedb9312306" containerName="payload-pre-processing" containerID="cri-o://f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff" gracePeriod=30
Jul 29 01:36:19.407443 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.407293    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:19.509554 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.509477    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:19.609487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.609462    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:19.618592 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.618567    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:36:19.656424 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.656395    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:36:19.665770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.665745    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:36:19.722001 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.721953    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/096be4f7-6eb2-421a-bdeb-82022493eba3-openshift-service-ca\") pod \"096be4f7-6eb2-421a-bdeb-82022493eba3\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") "
Jul 29 01:36:19.722186 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.722032    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/096be4f7-6eb2-421a-bdeb-82022493eba3-maas-api-tls\") pod \"096be4f7-6eb2-421a-bdeb-82022493eba3\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") "
Jul 29 01:36:19.722186 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.722118    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f946\" (UniqueName: \"kubernetes.io/projected/096be4f7-6eb2-421a-bdeb-82022493eba3-kube-api-access-2f946\") pod \"096be4f7-6eb2-421a-bdeb-82022493eba3\" (UID: \"096be4f7-6eb2-421a-bdeb-82022493eba3\") "
Jul 29 01:36:19.722327 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.722301    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/096be4f7-6eb2-421a-bdeb-82022493eba3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "096be4f7-6eb2-421a-bdeb-82022493eba3" (UID: "096be4f7-6eb2-421a-bdeb-82022493eba3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:19.722435 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.722415    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/096be4f7-6eb2-421a-bdeb-82022493eba3-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.724119 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.724093    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/096be4f7-6eb2-421a-bdeb-82022493eba3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "096be4f7-6eb2-421a-bdeb-82022493eba3" (UID: "096be4f7-6eb2-421a-bdeb-82022493eba3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:36:19.724220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.724157    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096be4f7-6eb2-421a-bdeb-82022493eba3-kube-api-access-2f946" (OuterVolumeSpecName: "kube-api-access-2f946") pod "096be4f7-6eb2-421a-bdeb-82022493eba3" (UID: "096be4f7-6eb2-421a-bdeb-82022493eba3"). InnerVolumeSpecName "kube-api-access-2f946". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:19.823626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.823539    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-plugins-config-volume\") pod \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") "
Jul 29 01:36:19.823626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.823592    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/573aa670-a87e-4b53-b7f5-cdedb9312306-plugins-config-volume\") pod \"573aa670-a87e-4b53-b7f5-cdedb9312306\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") "
Jul 29 01:36:19.823626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.823619    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc9np\" (UniqueName: \"kubernetes.io/projected/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-kube-api-access-cc9np\") pod \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\" (UID: \"fb129efc-ebc4-4bee-a8cb-6a5d3316598b\") "
Jul 29 01:36:19.823903 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.823749    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqx4p\" (UniqueName: \"kubernetes.io/projected/573aa670-a87e-4b53-b7f5-cdedb9312306-kube-api-access-hqx4p\") pod \"573aa670-a87e-4b53-b7f5-cdedb9312306\" (UID: \"573aa670-a87e-4b53-b7f5-cdedb9312306\") "
Jul 29 01:36:19.823979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.823955    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fb129efc-ebc4-4bee-a8cb-6a5d3316598b" (UID: "fb129efc-ebc4-4bee-a8cb-6a5d3316598b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:19.824037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.823970    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/573aa670-a87e-4b53-b7f5-cdedb9312306-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "573aa670-a87e-4b53-b7f5-cdedb9312306" (UID: "573aa670-a87e-4b53-b7f5-cdedb9312306"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:19.824105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.824045    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/096be4f7-6eb2-421a-bdeb-82022493eba3-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.824105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.824062    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.824105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.824073    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f946\" (UniqueName: \"kubernetes.io/projected/096be4f7-6eb2-421a-bdeb-82022493eba3-kube-api-access-2f946\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.824105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.824085    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/573aa670-a87e-4b53-b7f5-cdedb9312306-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.825743 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.825722    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-kube-api-access-cc9np" (OuterVolumeSpecName: "kube-api-access-cc9np") pod "fb129efc-ebc4-4bee-a8cb-6a5d3316598b" (UID: "fb129efc-ebc4-4bee-a8cb-6a5d3316598b"). InnerVolumeSpecName "kube-api-access-cc9np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:19.825804 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.825741    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573aa670-a87e-4b53-b7f5-cdedb9312306-kube-api-access-hqx4p" (OuterVolumeSpecName: "kube-api-access-hqx4p") pod "573aa670-a87e-4b53-b7f5-cdedb9312306" (UID: "573aa670-a87e-4b53-b7f5-cdedb9312306"). InnerVolumeSpecName "kube-api-access-hqx4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:19.885060 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.885025    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:19.924715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.924687    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqx4p\" (UniqueName: \"kubernetes.io/projected/573aa670-a87e-4b53-b7f5-cdedb9312306-kube-api-access-hqx4p\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.924715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.924715    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc9np\" (UniqueName: \"kubernetes.io/projected/fb129efc-ebc4-4bee-a8cb-6a5d3316598b-kube-api-access-cc9np\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:19.947165 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.947133    2555 generic.go:356] "Generic (PLEG): container finished" podID="fb129efc-ebc4-4bee-a8cb-6a5d3316598b" containerID="794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced" exitCode=0
Jul 29 01:36:19.947341 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.947179    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"
Jul 29 01:36:19.947341 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.947211    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw" event={"ID":"fb129efc-ebc4-4bee-a8cb-6a5d3316598b","Type":"ContainerDied","Data":"794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced"}
Jul 29 01:36:19.947341 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.947263    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw" event={"ID":"fb129efc-ebc4-4bee-a8cb-6a5d3316598b","Type":"ContainerDied","Data":"5a43748af73373667e1f5b33f4c98a1f13726d5ddd201567bae1a21b74145867"}
Jul 29 01:36:19.947526 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.947310    2555 scope.go:117] "RemoveContainer" containerID="794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced"
Jul 29 01:36:19.948713 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.948687    2555 generic.go:356] "Generic (PLEG): container finished" podID="573aa670-a87e-4b53-b7f5-cdedb9312306" containerID="f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff" exitCode=0
Jul 29 01:36:19.948821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.948729    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"
Jul 29 01:36:19.948821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.948774    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg" event={"ID":"573aa670-a87e-4b53-b7f5-cdedb9312306","Type":"ContainerDied","Data":"f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff"}
Jul 29 01:36:19.948926 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.948836    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg" event={"ID":"573aa670-a87e-4b53-b7f5-cdedb9312306","Type":"ContainerDied","Data":"7b92e8509b8f007eaf2fab1fc005812b33796108e80d0b2eeae13534c1b9adb9"}
Jul 29 01:36:19.950371 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.950336    2555 generic.go:356] "Generic (PLEG): container finished" podID="096be4f7-6eb2-421a-bdeb-82022493eba3" containerID="ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1" exitCode=0
Jul 29 01:36:19.950472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.950383    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"
Jul 29 01:36:19.950472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.950385    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc" event={"ID":"096be4f7-6eb2-421a-bdeb-82022493eba3","Type":"ContainerDied","Data":"ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1"}
Jul 29 01:36:19.950472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.950433    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc" event={"ID":"096be4f7-6eb2-421a-bdeb-82022493eba3","Type":"ContainerDied","Data":"86dff26d126e129b8116e29b06a37b17d50d1af1e600e4353a6d91a9b8798559"}
Jul 29 01:36:19.957006 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.956975    2555 scope.go:117] "RemoveContainer" containerID="794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced"
Jul 29 01:36:19.957327 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:36:19.957307    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced\": container with ID starting with 794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced not found: ID does not exist" containerID="794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced"
Jul 29 01:36:19.957418 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.957334    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced"} err="failed to get container status \"794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced\": rpc error: code = NotFound desc = could not find container \"794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced\": container with ID starting with 794b1cb898f7bc8814a690f6bdd22a581c54e2cc144a6e5a078fccf7ebf25ced not found: ID does not exist"
Jul 29 01:36:19.957418 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.957349    2555 scope.go:117] "RemoveContainer" containerID="f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff"
Jul 29 01:36:19.965768 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.965752    2555 scope.go:117] "RemoveContainer" containerID="f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff"
Jul 29 01:36:19.965987 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:36:19.965969    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff\": container with ID starting with f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff not found: ID does not exist" containerID="f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff"
Jul 29 01:36:19.966031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.965993    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff"} err="failed to get container status \"f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff\": rpc error: code = NotFound desc = could not find container \"f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff\": container with ID starting with f9f6fd168649c97be87f6826f787293352adbf5a64e3653f85c8bc4eddecd8ff not found: ID does not exist"
Jul 29 01:36:19.966031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.966009    2555 scope.go:117] "RemoveContainer" containerID="ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1"
Jul 29 01:36:19.973134 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.973108    2555 scope.go:117] "RemoveContainer" containerID="ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1"
Jul 29 01:36:19.973438 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:36:19.973419    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1\": container with ID starting with ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1 not found: ID does not exist" containerID="ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1"
Jul 29 01:36:19.973540 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.973443    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1"} err="failed to get container status \"ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1\": rpc error: code = NotFound desc = could not find container \"ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1\": container with ID starting with ec7260463bebcba8c6c75f6a50ad098684813d772d34c1262803058a4252e6c1 not found: ID does not exist"
Jul 29 01:36:19.976391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.976369    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"]
Jul 29 01:36:19.981909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.981888    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6afea2-ccd9f988-wsblw"]
Jul 29 01:36:19.992341 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.992323    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"]
Jul 29 01:36:19.999394 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:19.996938    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6afea2-554fc4744d-r97rg"]
Jul 29 01:36:20.007851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:20.007825    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"]
Jul 29 01:36:20.013134 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:20.013104    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6afea2-cd5bdbfb8-b7dkc"]
Jul 29 01:36:21.190695 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:21.190660    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="096be4f7-6eb2-421a-bdeb-82022493eba3" path="/var/lib/kubelet/pods/096be4f7-6eb2-421a-bdeb-82022493eba3/volumes"
Jul 29 01:36:21.191076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:21.191014    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="573aa670-a87e-4b53-b7f5-cdedb9312306" path="/var/lib/kubelet/pods/573aa670-a87e-4b53-b7f5-cdedb9312306/volumes"
Jul 29 01:36:21.191339 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:21.191325    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb129efc-ebc4-4bee-a8cb-6a5d3316598b" path="/var/lib/kubelet/pods/fb129efc-ebc4-4bee-a8cb-6a5d3316598b/volumes"
Jul 29 01:36:25.666633 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:25.666555    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:25.733355 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:25.733317    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:25.932460 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:25.932377    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:26.185565 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:26.185480    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:36.877017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.876981    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"]
Jul 29 01:36:36.877465 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.877245    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" podUID="a8f0ccb8-5199-45a8-ae25-99bf183f1a93" containerName="maas-api" containerID="cri-o://ab9c70de3a7866220a4eecefa65ee617c43e52e7c579b6db9194a502bba0f32f" gracePeriod=30
Jul 29 01:36:36.906721 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.906691    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:36.926410 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.926380    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"]
Jul 29 01:36:36.926649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.926624    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj" podUID="7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" containerName="payload-processing" containerID="cri-o://6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3" gracePeriod=30
Jul 29 01:36:36.931568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.931043    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"]
Jul 29 01:36:36.931568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:36.931319    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp" podUID="26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" containerName="payload-pre-processing" containerID="cri-o://8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463" gracePeriod=30
Jul 29 01:36:37.013400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.013361    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:37.042474 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.042408    2555 generic.go:356] "Generic (PLEG): container finished" podID="a8f0ccb8-5199-45a8-ae25-99bf183f1a93" containerID="ab9c70de3a7866220a4eecefa65ee617c43e52e7c579b6db9194a502bba0f32f" exitCode=0
Jul 29 01:36:37.042539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.042488    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" event={"ID":"a8f0ccb8-5199-45a8-ae25-99bf183f1a93","Type":"ContainerDied","Data":"ab9c70de3a7866220a4eecefa65ee617c43e52e7c579b6db9194a502bba0f32f"}
Jul 29 01:36:37.082394 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.082352    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:37.259373 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.259345    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:36:37.262270 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.262249    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:36:37.265551 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.265530    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:36:37.288046 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.288017    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-openshift-service-ca\") pod \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") "
Jul 29 01:36:37.288238 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.288063    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx9xt\" (UniqueName: \"kubernetes.io/projected/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-kube-api-access-mx9xt\") pod \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") "
Jul 29 01:36:37.288238 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.288108    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls\") pod \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\" (UID: \"a8f0ccb8-5199-45a8-ae25-99bf183f1a93\") "
Jul 29 01:36:37.288440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.288394    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8f0ccb8-5199-45a8-ae25-99bf183f1a93" (UID: "a8f0ccb8-5199-45a8-ae25-99bf183f1a93"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:37.290231 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.290207    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8f0ccb8-5199-45a8-ae25-99bf183f1a93" (UID: "a8f0ccb8-5199-45a8-ae25-99bf183f1a93"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:36:37.290562 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.290249    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-kube-api-access-mx9xt" (OuterVolumeSpecName: "kube-api-access-mx9xt") pod "a8f0ccb8-5199-45a8-ae25-99bf183f1a93" (UID: "a8f0ccb8-5199-45a8-ae25-99bf183f1a93"). InnerVolumeSpecName "kube-api-access-mx9xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:37.388739 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.388701    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf5wz\" (UniqueName: \"kubernetes.io/projected/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-kube-api-access-pf5wz\") pod \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") "
Jul 29 01:36:37.388914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.388783    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-plugins-config-volume\") pod \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") "
Jul 29 01:36:37.388914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.388808    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr8k8\" (UniqueName: \"kubernetes.io/projected/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-kube-api-access-rr8k8\") pod \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\" (UID: \"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6\") "
Jul 29 01:36:37.388914 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.388835    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-plugins-config-volume\") pod \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\" (UID: \"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e\") "
Jul 29 01:36:37.389112 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.389048    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx9xt\" (UniqueName: \"kubernetes.io/projected/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-kube-api-access-mx9xt\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.389112 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.389071    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.389112 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.389085    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f0ccb8-5199-45a8-ae25-99bf183f1a93-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.389314 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.389232    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" (UID: "26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:37.389376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.389311    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" (UID: "7959ec2f-edc7-4f17-9ec1-72dbb6708a0e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:36:37.390934 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.390908    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-kube-api-access-rr8k8" (OuterVolumeSpecName: "kube-api-access-rr8k8") pod "26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" (UID: "26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6"). InnerVolumeSpecName "kube-api-access-rr8k8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:37.391043 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.390932    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-kube-api-access-pf5wz" (OuterVolumeSpecName: "kube-api-access-pf5wz") pod "7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" (UID: "7959ec2f-edc7-4f17-9ec1-72dbb6708a0e"). InnerVolumeSpecName "kube-api-access-pf5wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:36:37.490194 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.490157    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf5wz\" (UniqueName: \"kubernetes.io/projected/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-kube-api-access-pf5wz\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.490194 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.490192    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.490436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.490202    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr8k8\" (UniqueName: \"kubernetes.io/projected/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6-kube-api-access-rr8k8\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.490436 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.490215    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:36:37.692632 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:37.692598    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:38.049317 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.049271    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"
Jul 29 01:36:38.049810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.049322    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh" event={"ID":"a8f0ccb8-5199-45a8-ae25-99bf183f1a93","Type":"ContainerDied","Data":"882f5cac6348d61800b38c5fafc33ac8d5698bfd60ec7d86ed69d93181d10128"}
Jul 29 01:36:38.049810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.049784    2555 scope.go:117] "RemoveContainer" containerID="ab9c70de3a7866220a4eecefa65ee617c43e52e7c579b6db9194a502bba0f32f"
Jul 29 01:36:38.050857 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.050826    2555 generic.go:356] "Generic (PLEG): container finished" podID="26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" containerID="8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463" exitCode=0
Jul 29 01:36:38.051012 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.050908    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"
Jul 29 01:36:38.051012 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.050914    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp" event={"ID":"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6","Type":"ContainerDied","Data":"8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463"}
Jul 29 01:36:38.051012 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.050956    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp" event={"ID":"26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6","Type":"ContainerDied","Data":"9c4a9a566b48dfbe397036ab92dcd4aa7c6477513bc1c66f36e619e1317bc994"}
Jul 29 01:36:38.052434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.052406    2555 generic.go:356] "Generic (PLEG): container finished" podID="7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" containerID="6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3" exitCode=0
Jul 29 01:36:38.052551 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.052438    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"
Jul 29 01:36:38.052551 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.052463    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj" event={"ID":"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e","Type":"ContainerDied","Data":"6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3"}
Jul 29 01:36:38.052551 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.052502    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj" event={"ID":"7959ec2f-edc7-4f17-9ec1-72dbb6708a0e","Type":"ContainerDied","Data":"894d69d86ce3a6ebbb1c5dd75e357594db0f0e79b93cbcb5f13f415646bafc7d"}
Jul 29 01:36:38.061809 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.061792    2555 scope.go:117] "RemoveContainer" containerID="8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463"
Jul 29 01:36:38.070356 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.070338    2555 scope.go:117] "RemoveContainer" containerID="8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463"
Jul 29 01:36:38.070637 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:36:38.070615    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463\": container with ID starting with 8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463 not found: ID does not exist" containerID="8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463"
Jul 29 01:36:38.070717 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.070643    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463"} err="failed to get container status \"8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463\": rpc error: code = NotFound desc = could not find container \"8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463\": container with ID starting with 8397f7d16af15f1900c74c86f47721605513ebab9b1bc262ef244e68510fd463 not found: ID does not exist"
Jul 29 01:36:38.070717 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.070659    2555 scope.go:117] "RemoveContainer" containerID="6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3"
Jul 29 01:36:38.074207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.074185    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"]
Jul 29 01:36:38.078308 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.078265    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-f5ec11-566575dc79-2cqvh"]
Jul 29 01:36:38.079574 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.079555    2555 scope.go:117] "RemoveContainer" containerID="6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3"
Jul 29 01:36:38.079837 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:36:38.079816    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3\": container with ID starting with 6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3 not found: ID does not exist" containerID="6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3"
Jul 29 01:36:38.079885 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.079844    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3"} err="failed to get container status \"6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3\": rpc error: code = NotFound desc = could not find container \"6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3\": container with ID starting with 6f68f33f59efc65201012de7e820cf37d3b5656869e25c2e92927497f914a4f3 not found: ID does not exist"
Jul 29 01:36:38.088725 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.088704    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"]
Jul 29 01:36:38.092865 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.092845    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-f5ec11-5ff67d8c94-pnktj"]
Jul 29 01:36:38.103578 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.103556    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"]
Jul 29 01:36:38.107671 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:38.107649    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-f5ec11-584bdc8bb4-rftpp"]
Jul 29 01:36:39.190543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:39.190511    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" path="/var/lib/kubelet/pods/26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6/volumes"
Jul 29 01:36:39.190920 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:39.190865    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" path="/var/lib/kubelet/pods/7959ec2f-edc7-4f17-9ec1-72dbb6708a0e/volumes"
Jul 29 01:36:39.191205 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:39.191192    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8f0ccb8-5199-45a8-ae25-99bf183f1a93" path="/var/lib/kubelet/pods/a8f0ccb8-5199-45a8-ae25-99bf183f1a93/volumes"
Jul 29 01:36:43.673268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:43.673229    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:43.773888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:43.773851    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:43.939535 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:43.939504    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:44.216357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.216242    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:44.540153 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.540062    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:44.827179 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827091    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"]
Jul 29 01:36:44.827610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827556    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096be4f7-6eb2-421a-bdeb-82022493eba3" containerName="maas-api"
Jul 29 01:36:44.827610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827571    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="096be4f7-6eb2-421a-bdeb-82022493eba3" containerName="maas-api"
Jul 29 01:36:44.827610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827580    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573aa670-a87e-4b53-b7f5-cdedb9312306" containerName="payload-pre-processing"
Jul 29 01:36:44.827610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827586    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="573aa670-a87e-4b53-b7f5-cdedb9312306" containerName="payload-pre-processing"
Jul 29 01:36:44.827610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827594    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" containerName="payload-processing"
Jul 29 01:36:44.827610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827600    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" containerName="payload-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827614    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8f0ccb8-5199-45a8-ae25-99bf183f1a93" containerName="maas-api"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827622    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8f0ccb8-5199-45a8-ae25-99bf183f1a93" containerName="maas-api"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827632    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb129efc-ebc4-4bee-a8cb-6a5d3316598b" containerName="payload-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827638    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb129efc-ebc4-4bee-a8cb-6a5d3316598b" containerName="payload-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827647    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" containerName="payload-pre-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827652    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" containerName="payload-pre-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827718    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb129efc-ebc4-4bee-a8cb-6a5d3316598b" containerName="payload-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827727    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="096be4f7-6eb2-421a-bdeb-82022493eba3" containerName="maas-api"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827733    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="573aa670-a87e-4b53-b7f5-cdedb9312306" containerName="payload-pre-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827739    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7959ec2f-edc7-4f17-9ec1-72dbb6708a0e" containerName="payload-processing"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827747    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8f0ccb8-5199-45a8-ae25-99bf183f1a93" containerName="maas-api"
Jul 29 01:36:44.827814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.827753    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="26ba9cb5-bce3-4bc1-b7df-5520d56b3cf6" containerName="payload-pre-processing"
Jul 29 01:36:44.835922 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.835900    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.838759 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.838738    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-aa4526-openshift-default-dockercfg-xm2s6\""
Jul 29 01:36:44.838893 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.838747    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 01:36:44.844745 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.844717    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"]
Jul 29 01:36:44.859012 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.858982    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:44.957872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.957839    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-envoy\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.957877    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf5j8\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-kube-api-access-kf5j8\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.957901    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd6026c-e1af-452c-823d-0077e854deef-istiod-ca-cert\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.957926    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-credential-socket\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.957945    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-istio-token\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.958005    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd6026c-e1af-452c-823d-0077e854deef-istio-podinfo\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958037 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.958028    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-socket\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958253 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.958048    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-certs\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:44.958253 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:44.958065    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-data\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.039983 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.039947    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:45.059262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059223    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-envoy\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059264    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf5j8\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-kube-api-access-kf5j8\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059301    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd6026c-e1af-452c-823d-0077e854deef-istiod-ca-cert\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059328    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-credential-socket\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059350    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-istio-token\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059407    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd6026c-e1af-452c-823d-0077e854deef-istio-podinfo\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059436    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-socket\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059456    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-certs\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059490    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-data\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059779    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-credential-socket\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059889    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-certs\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.059940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.059919    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-data\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.060186 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.060154    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd6026c-e1af-452c-823d-0077e854deef-istiod-ca-cert\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.060248 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.060225    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-socket\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.061586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.061568    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-envoy\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.061857 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.061839    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd6026c-e1af-452c-823d-0077e854deef-istio-podinfo\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.069116 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.069091    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-istio-token\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.069224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.069147    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf5j8\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-kube-api-access-kf5j8\") pod \"e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") " pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.149174 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.149071    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:45.296141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.296119    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"]
Jul 29 01:36:45.298472 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:36:45.298445    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd6026c_e1af_452c_823d_0077e854deef.slice/crio-f2d7b5b81465455ebaffdec5e3c02edf9bdd955826cc2a1f038ecb873a1de06d WatchSource:0}: Error finding container f2d7b5b81465455ebaffdec5e3c02edf9bdd955826cc2a1f038ecb873a1de06d: Status 404 returned error can't find the container with id f2d7b5b81465455ebaffdec5e3c02edf9bdd955826cc2a1f038ecb873a1de06d
Jul 29 01:36:45.300713 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:45.300698    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:36:46.099697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:46.099658    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" event={"ID":"fcd6026c-e1af-452c-823d-0077e854deef","Type":"ContainerStarted","Data":"f2d7b5b81465455ebaffdec5e3c02edf9bdd955826cc2a1f038ecb873a1de06d"}
Jul 29 01:36:49.189191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:49.186952    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:36:49.189191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:49.187239    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:36:49.189191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:49.187443    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:36:50.124946 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:50.124908    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" event={"ID":"fcd6026c-e1af-452c-823d-0077e854deef","Type":"ContainerStarted","Data":"79575e1b15e33fa33d61f67986f9c0fef7d83de282695c7f17480ebeee93062f"}
Jul 29 01:36:50.149910 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:50.149850    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" podStartSLOduration=2.264342088 podStartE2EDuration="6.149832521s" totalImagesPullingTime="3.885490433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:36:50.147646698 +0000 UTC m=+2539.478296305" watchObservedRunningTime="2026-07-29 01:36:50.149832521 +0000 UTC m=+2539.480482128"
Jul 29 01:36:50.150074 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:50.149956    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:50.154769 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:50.154744    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:51.130799 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:51.130769    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:51.131798 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:51.131770    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:36:52.093089 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.093056    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:52.167610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.167580    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"]
Jul 29 01:36:52.171132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.171110    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.171711 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.171685    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:52.174207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.174185    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-aa4526-serving-cert\""
Jul 29 01:36:52.183992 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.183972    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"]
Jul 29 01:36:52.239585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.239559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:52.332410 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.332371    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92be1b3c-1576-4df3-a07f-9dc398358546-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.332566 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.332465    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmgk6\" (UniqueName: \"kubernetes.io/projected/92be1b3c-1576-4df3-a07f-9dc398358546-kube-api-access-kmgk6\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.332566 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.332536    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92be1b3c-1576-4df3-a07f-9dc398358546-maas-api-tls\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.341291 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.341247    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:52.433306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.433193    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmgk6\" (UniqueName: \"kubernetes.io/projected/92be1b3c-1576-4df3-a07f-9dc398358546-kube-api-access-kmgk6\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.433306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.433256    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92be1b3c-1576-4df3-a07f-9dc398358546-maas-api-tls\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.433546 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.433328    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92be1b3c-1576-4df3-a07f-9dc398358546-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.433896 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.433874    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92be1b3c-1576-4df3-a07f-9dc398358546-openshift-service-ca\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.435653 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.435635    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92be1b3c-1576-4df3-a07f-9dc398358546-maas-api-tls\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.443681 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.443652    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmgk6\" (UniqueName: \"kubernetes.io/projected/92be1b3c-1576-4df3-a07f-9dc398358546-kube-api-access-kmgk6\") pod \"maas-api-e2e-api-a-aa4526-764bf865f-sf6vw\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.481796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.481775    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:52.638799 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.638771    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"]
Jul 29 01:36:52.640640 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:36:52.640612    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92be1b3c_1576_4df3_a07f_9dc398358546.slice/crio-18557c6c35a980bfa2bbaaf9c59eae22b0d9fc41c9e71218f274950fbd45f7bb WatchSource:0}: Error finding container 18557c6c35a980bfa2bbaaf9c59eae22b0d9fc41c9e71218f274950fbd45f7bb: Status 404 returned error can't find the container with id 18557c6c35a980bfa2bbaaf9c59eae22b0d9fc41c9e71218f274950fbd45f7bb
Jul 29 01:36:52.921503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:52.921466    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:53.026028 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.025995    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"]
Jul 29 01:36:53.031015 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.030988    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.033957 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.033931    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-aa4526\""
Jul 29 01:36:53.044723 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.044694    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"]
Jul 29 01:36:53.059541 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.059512    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"]
Jul 29 01:36:53.063866 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.063844    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.067099 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.067077    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-aa4526-dockercfg-bdrsj\""
Jul 29 01:36:53.072187 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.072166    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"]
Jul 29 01:36:53.080330 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.080265    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:36:53.140923 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.140887    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.141117 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.140939    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkndh\" (UniqueName: \"kubernetes.io/projected/b3755f62-e630-42ea-85db-fd9a2adc01eb-kube-api-access-dkndh\") pod \"payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.141117 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.141020    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3755f62-e630-42ea-85db-fd9a2adc01eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.141231 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.141112    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6mkx\" (UniqueName: \"kubernetes.io/projected/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-kube-api-access-t6mkx\") pod \"payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.143563 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.143532    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw" event={"ID":"92be1b3c-1576-4df3-a07f-9dc398358546","Type":"ContainerStarted","Data":"18557c6c35a980bfa2bbaaf9c59eae22b0d9fc41c9e71218f274950fbd45f7bb"}
Jul 29 01:36:53.241917 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.241866    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.242413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.241926    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkndh\" (UniqueName: \"kubernetes.io/projected/b3755f62-e630-42ea-85db-fd9a2adc01eb-kube-api-access-dkndh\") pod \"payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.242413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.241965    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3755f62-e630-42ea-85db-fd9a2adc01eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.242413 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.242021    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6mkx\" (UniqueName: \"kubernetes.io/projected/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-kube-api-access-t6mkx\") pod \"payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.242923 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.242893    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.243140 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.243117    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3755f62-e630-42ea-85db-fd9a2adc01eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.252016 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.251978    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6mkx\" (UniqueName: \"kubernetes.io/projected/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-kube-api-access-t6mkx\") pod \"payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.252252 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.252230    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkndh\" (UniqueName: \"kubernetes.io/projected/b3755f62-e630-42ea-85db-fd9a2adc01eb-kube-api-access-dkndh\") pod \"payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.343199 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.343168    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:53.375680 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.375651    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:53.498868 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.498839    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"]
Jul 29 01:36:53.499557 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:36:53.499532    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ad165d_9e24_4aba_9ff9_b06c8cee02ca.slice/crio-6f25bb530e6d0f9f5739c6ddd61f07442ac40b5a08197f56dbbe9050ba4de232 WatchSource:0}: Error finding container 6f25bb530e6d0f9f5739c6ddd61f07442ac40b5a08197f56dbbe9050ba4de232: Status 404 returned error can't find the container with id 6f25bb530e6d0f9f5739c6ddd61f07442ac40b5a08197f56dbbe9050ba4de232
Jul 29 01:36:53.539474 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:53.539426    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"]
Jul 29 01:36:53.544492 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:36:53.544455    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3755f62_e630_42ea_85db_fd9a2adc01eb.slice/crio-0a33ef8e6b455f387ba78b17f4971b8b8a021a520db27343203733ec4daf03b1 WatchSource:0}: Error finding container 0a33ef8e6b455f387ba78b17f4971b8b8a021a520db27343203733ec4daf03b1: Status 404 returned error can't find the container with id 0a33ef8e6b455f387ba78b17f4971b8b8a021a520db27343203733ec4daf03b1
Jul 29 01:36:54.151605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.151569    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw" event={"ID":"92be1b3c-1576-4df3-a07f-9dc398358546","Type":"ContainerStarted","Data":"0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063"}
Jul 29 01:36:54.151806 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.151697    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:36:54.153164 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.153142    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5" event={"ID":"b3755f62-e630-42ea-85db-fd9a2adc01eb","Type":"ContainerStarted","Data":"3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8"}
Jul 29 01:36:54.153313 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.153177    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5" event={"ID":"b3755f62-e630-42ea-85db-fd9a2adc01eb","Type":"ContainerStarted","Data":"0a33ef8e6b455f387ba78b17f4971b8b8a021a520db27343203733ec4daf03b1"}
Jul 29 01:36:54.153313 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.153263    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:36:54.154639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.154616    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc" event={"ID":"84ad165d-9e24-4aba-9ff9-b06c8cee02ca","Type":"ContainerStarted","Data":"e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0"}
Jul 29 01:36:54.154735 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.154663    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:36:54.154735 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.154680    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc" event={"ID":"84ad165d-9e24-4aba-9ff9-b06c8cee02ca","Type":"ContainerStarted","Data":"6f25bb530e6d0f9f5739c6ddd61f07442ac40b5a08197f56dbbe9050ba4de232"}
Jul 29 01:36:54.174284 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.174225    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw" podStartSLOduration=1.219306296 podStartE2EDuration="2.174212099s" totalImagesPullingTime="954.905803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:36:54.17122461 +0000 UTC m=+2543.501874228" watchObservedRunningTime="2026-07-29 01:36:54.174212099 +0000 UTC m=+2543.504861704"
Jul 29 01:36:54.192071 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.192029    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5" podStartSLOduration=1.19201698 podStartE2EDuration="1.19201698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:36:54.19013421 +0000 UTC m=+2543.520783818" watchObservedRunningTime="2026-07-29 01:36:54.19201698 +0000 UTC m=+2543.522666585"
Jul 29 01:36:54.208359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:36:54.208318    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc" podStartSLOduration=1.208304213 podStartE2EDuration="1.208304213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:36:54.206715038 +0000 UTC m=+2543.537364657" watchObservedRunningTime="2026-07-29 01:36:54.208304213 +0000 UTC m=+2543.538953819"
Jul 29 01:37:00.166191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:00.166159    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:37:05.162847 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.162816    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:37:05.163349 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.162884    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:37:05.215410 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.215374    2555 generic.go:356] "Generic (PLEG): container finished" podID="fcd6026c-e1af-452c-823d-0077e854deef" containerID="79575e1b15e33fa33d61f67986f9c0fef7d83de282695c7f17480ebeee93062f" exitCode=0
Jul 29 01:37:05.215410 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.215402    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" event={"ID":"fcd6026c-e1af-452c-823d-0077e854deef","Type":"ContainerDied","Data":"79575e1b15e33fa33d61f67986f9c0fef7d83de282695c7f17480ebeee93062f"}
Jul 29 01:37:05.215845 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.215830    2555 scope.go:117] "RemoveContainer" containerID="79575e1b15e33fa33d61f67986f9c0fef7d83de282695c7f17480ebeee93062f"
Jul 29 01:37:05.216951 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.216923    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:05.217033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.216992    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:05.217033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:05.217025    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:06.131517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:06.131470    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:37:06.222720 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:06.222685    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" event={"ID":"fcd6026c-e1af-452c-823d-0077e854deef","Type":"ContainerStarted","Data":"77b920ae2c993465be7628d337bea352b81c29170d867962054baef663f11dee"}
Jul 29 01:37:07.149598 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:07.149557    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:37:07.150827 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:07.150791    2555 patch_prober.go:28] interesting pod/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 01:37:07.150972 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:07.150870    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 01:37:08.149934 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:08.149886    2555 patch_prober.go:28] interesting pod/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 01:37:08.150539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:08.149959    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 01:37:09.150518 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:09.150473    2555 patch_prober.go:28] interesting pod/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Jul 29 01:37:09.151029 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:09.150580    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Jul 29 01:37:10.154152 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:10.154115    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:37:10.154647 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:10.154536    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:37:10.155160 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:10.155141    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:37:17.000217 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.000181    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:17.459441 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.459405    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:17.745764 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.745673    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"]
Jul 29 01:37:17.751684 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.751663    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.754726 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.754701    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4126fe-openshift-default-dockercfg-bvkvg\""
Jul 29 01:37:17.764625 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.764604    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"]
Jul 29 01:37:17.770217 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.770189    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:17.874563 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874524    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-envoy\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874563 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874564    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-data\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874595    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-token\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874661    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-socket\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874726    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-podinfo\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874756    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26dn\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-kube-api-access-z26dn\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874795    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-credential-socket\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874847    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1950c97-7a58-4c42-9176-d779f6dad1eb-istiod-ca-cert\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.874991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.874881    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-certs\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.920506 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.920474    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:17.975943 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.975906    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-podinfo\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.975943 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.975942    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26dn\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-kube-api-access-z26dn\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976083    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-credential-socket\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976139    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1950c97-7a58-4c42-9176-d779f6dad1eb-istiod-ca-cert\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976206    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-certs\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976357 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976244    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-envoy\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976471 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976350    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-data\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976471 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976409    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-token\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976472    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-socket\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976692 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976502    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-credential-socket\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976640    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-certs\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976763 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976738    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-data\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976874 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976802    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-socket\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.976874 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.976860    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1950c97-7a58-4c42-9176-d779f6dad1eb-istiod-ca-cert\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.978761 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.978733    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-envoy\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.979012 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.978990    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-podinfo\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.984296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.984246    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-token\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:17.984428 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:17.984411    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26dn\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-kube-api-access-z26dn\") pod \"e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") " pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:18.062472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.062383    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:18.213972 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.212955    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"]
Jul 29 01:37:18.222638 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.222602    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:18.222804 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.222684    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:18.222804 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.222728    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:18.292031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.291995    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" event={"ID":"b1950c97-7a58-4c42-9176-d779f6dad1eb","Type":"ContainerStarted","Data":"f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"}
Jul 29 01:37:18.292192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.292054    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" event={"ID":"b1950c97-7a58-4c42-9176-d779f6dad1eb","Type":"ContainerStarted","Data":"12245d9921bb90e4c797bc2087c3c7f67f3295e07598d2b1055ba4fb3baadb11"}
Jul 29 01:37:18.315533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:18.315431    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podStartSLOduration=1.315414377 podStartE2EDuration="1.315414377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:37:18.313316753 +0000 UTC m=+2567.643966359" watchObservedRunningTime="2026-07-29 01:37:18.315414377 +0000 UTC m=+2567.646063983"
Jul 29 01:37:19.063263 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:19.063221    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:19.067919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:19.067894    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:19.298482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:19.298452    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:19.299519 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:19.299501    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:20.197518 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:20.197481    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:20.290328 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:20.290260    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:20.378139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:20.378086    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:20.620726 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:20.620649    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:21.009639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.009606    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:21.110849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.110814    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:21.152256 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.152204    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"]
Jul 29 01:37:21.156529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.156504    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.159551 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.159527    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-4126fe-dockercfg-4nv8p\""
Jul 29 01:37:21.159668 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.159531    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-4126fe\""
Jul 29 01:37:21.168492 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.168469    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"]
Jul 29 01:37:21.209169 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.209131    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.209544 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.209194    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z5vp\" (UniqueName: \"kubernetes.io/projected/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-kube-api-access-6z5vp\") pod \"payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.310166 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.310076    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.310166 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.310138    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z5vp\" (UniqueName: \"kubernetes.io/projected/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-kube-api-access-6z5vp\") pod \"payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.310886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.310857    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.320243 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.320211    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z5vp\" (UniqueName: \"kubernetes.io/projected/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-kube-api-access-6z5vp\") pod \"payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.468824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.468792    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:21.618336 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:21.618304    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"]
Jul 29 01:37:21.620465 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:37:21.620432    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7d6f47_ab4d_474c_b82d_6207c73a41f4.slice/crio-2ce2a73132eb529e3b7a2ee6a73b839312b899cbf18b8b2f03aa260b90bc5bde WatchSource:0}: Error finding container 2ce2a73132eb529e3b7a2ee6a73b839312b899cbf18b8b2f03aa260b90bc5bde: Status 404 returned error can't find the container with id 2ce2a73132eb529e3b7a2ee6a73b839312b899cbf18b8b2f03aa260b90bc5bde
Jul 29 01:37:22.316256 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:22.316217    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c" event={"ID":"7b7d6f47-ab4d-474c-b82d-6207c73a41f4","Type":"ContainerStarted","Data":"7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1"}
Jul 29 01:37:22.316762 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:22.316335    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c" event={"ID":"7b7d6f47-ab4d-474c-b82d-6207c73a41f4","Type":"ContainerStarted","Data":"2ce2a73132eb529e3b7a2ee6a73b839312b899cbf18b8b2f03aa260b90bc5bde"}
Jul 29 01:37:22.316762 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:22.316374    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:22.333379 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:22.333330    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c" podStartSLOduration=1.3333113349999999 podStartE2EDuration="1.333311335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:37:22.332623784 +0000 UTC m=+2571.663273393" watchObservedRunningTime="2026-07-29 01:37:22.333311335 +0000 UTC m=+2571.663960941"
Jul 29 01:37:30.440067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:30.440029    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:30.540645 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:30.540604    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:37:33.324908 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:33.324877    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:37:34.300610 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.300543    2555 patch_prober.go:28] interesting pod/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 29 01:37:34.300781 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.300606    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 29 01:37:34.388663 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.388632    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerID="f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf" exitCode=0
Jul 29 01:37:34.389051 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.388666    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" event={"ID":"b1950c97-7a58-4c42-9176-d779f6dad1eb","Type":"ContainerDied","Data":"f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"}
Jul 29 01:37:34.389051 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.388986    2555 scope.go:117] "RemoveContainer" containerID="f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"
Jul 29 01:37:34.389993 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.389964    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:34.390065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.390041    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:34.390104 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:34.390075    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:37:35.395892 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:35.395855    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" event={"ID":"b1950c97-7a58-4c42-9176-d779f6dad1eb","Type":"ContainerStarted","Data":"455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e"}
Jul 29 01:37:36.063356 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:36.063321    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:36.063574 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:36.063517    2555 patch_prober.go:28] interesting pod/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 29 01:37:36.063641 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:36.063566    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 29 01:37:37.063378 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:37.063345    2555 patch_prober.go:28] interesting pod/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 29 01:37:37.063759 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:37.063412    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 29 01:37:38.062897 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:38.062853    2555 patch_prober.go:28] interesting pod/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 29 01:37:38.063106 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:38.062927    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 29 01:37:39.063419 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:39.063370    2555 patch_prober.go:28] interesting pod/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Jul 29 01:37:39.063841 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:39.063447    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Jul 29 01:37:40.067183 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:40.067151    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:40.067601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:40.067495    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:40.068185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:40.068169    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:37:45.151082 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:37:45.151040    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:03.567581 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.567355    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"]
Jul 29 01:38:03.568391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.567711    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw" podUID="92be1b3c-1576-4df3-a07f-9dc398358546" containerName="maas-api" containerID="cri-o://0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063" gracePeriod=30
Jul 29 01:38:03.585419 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.585387    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:03.613973 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.613943    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"]
Jul 29 01:38:03.614194 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.614167    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5" podUID="b3755f62-e630-42ea-85db-fd9a2adc01eb" containerName="payload-processing" containerID="cri-o://3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8" gracePeriod=30
Jul 29 01:38:03.619772 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.619746    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"]
Jul 29 01:38:03.619971 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.619947    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc" podUID="84ad165d-9e24-4aba-9ff9-b06c8cee02ca" containerName="payload-pre-processing" containerID="cri-o://e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0" gracePeriod=30
Jul 29 01:38:03.669420 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.669391    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:03.731906 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.731808    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:03.796566 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.796451    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:03.882694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.882667    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:38:03.920479 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.920452    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:38:03.942899 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:03.942873    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:38:04.025010 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.024987    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92be1b3c-1576-4df3-a07f-9dc398358546-maas-api-tls\") pod \"92be1b3c-1576-4df3-a07f-9dc398358546\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") "
Jul 29 01:38:04.025195 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025037    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6mkx\" (UniqueName: \"kubernetes.io/projected/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-kube-api-access-t6mkx\") pod \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") "
Jul 29 01:38:04.025195 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025067    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmgk6\" (UniqueName: \"kubernetes.io/projected/92be1b3c-1576-4df3-a07f-9dc398358546-kube-api-access-kmgk6\") pod \"92be1b3c-1576-4df3-a07f-9dc398358546\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") "
Jul 29 01:38:04.025195 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025085    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-plugins-config-volume\") pod \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\" (UID: \"84ad165d-9e24-4aba-9ff9-b06c8cee02ca\") "
Jul 29 01:38:04.025195 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025128    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkndh\" (UniqueName: \"kubernetes.io/projected/b3755f62-e630-42ea-85db-fd9a2adc01eb-kube-api-access-dkndh\") pod \"b3755f62-e630-42ea-85db-fd9a2adc01eb\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") "
Jul 29 01:38:04.025195 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025161    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92be1b3c-1576-4df3-a07f-9dc398358546-openshift-service-ca\") pod \"92be1b3c-1576-4df3-a07f-9dc398358546\" (UID: \"92be1b3c-1576-4df3-a07f-9dc398358546\") "
Jul 29 01:38:04.025525 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025336    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3755f62-e630-42ea-85db-fd9a2adc01eb-plugins-config-volume\") pod \"b3755f62-e630-42ea-85db-fd9a2adc01eb\" (UID: \"b3755f62-e630-42ea-85db-fd9a2adc01eb\") "
Jul 29 01:38:04.025525 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025495    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92be1b3c-1576-4df3-a07f-9dc398358546-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92be1b3c-1576-4df3-a07f-9dc398358546" (UID: "92be1b3c-1576-4df3-a07f-9dc398358546"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:38:04.025627 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025526    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84ad165d-9e24-4aba-9ff9-b06c8cee02ca" (UID: "84ad165d-9e24-4aba-9ff9-b06c8cee02ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:38:04.025785 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025754    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92be1b3c-1576-4df3-a07f-9dc398358546-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.025785 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025778    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.025913 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.025813    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3755f62-e630-42ea-85db-fd9a2adc01eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3755f62-e630-42ea-85db-fd9a2adc01eb" (UID: "b3755f62-e630-42ea-85db-fd9a2adc01eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:38:04.027190 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.027165    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92be1b3c-1576-4df3-a07f-9dc398358546-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92be1b3c-1576-4df3-a07f-9dc398358546" (UID: "92be1b3c-1576-4df3-a07f-9dc398358546"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:38:04.027324 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.027268    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92be1b3c-1576-4df3-a07f-9dc398358546-kube-api-access-kmgk6" (OuterVolumeSpecName: "kube-api-access-kmgk6") pod "92be1b3c-1576-4df3-a07f-9dc398358546" (UID: "92be1b3c-1576-4df3-a07f-9dc398358546"). InnerVolumeSpecName "kube-api-access-kmgk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:04.027642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.027622    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-kube-api-access-t6mkx" (OuterVolumeSpecName: "kube-api-access-t6mkx") pod "84ad165d-9e24-4aba-9ff9-b06c8cee02ca" (UID: "84ad165d-9e24-4aba-9ff9-b06c8cee02ca"). InnerVolumeSpecName "kube-api-access-t6mkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:04.027727 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.027653    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3755f62-e630-42ea-85db-fd9a2adc01eb-kube-api-access-dkndh" (OuterVolumeSpecName: "kube-api-access-dkndh") pod "b3755f62-e630-42ea-85db-fd9a2adc01eb" (UID: "b3755f62-e630-42ea-85db-fd9a2adc01eb"). InnerVolumeSpecName "kube-api-access-dkndh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:04.126601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.126544    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92be1b3c-1576-4df3-a07f-9dc398358546-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.126601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.126567    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6mkx\" (UniqueName: \"kubernetes.io/projected/84ad165d-9e24-4aba-9ff9-b06c8cee02ca-kube-api-access-t6mkx\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.126601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.126578    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmgk6\" (UniqueName: \"kubernetes.io/projected/92be1b3c-1576-4df3-a07f-9dc398358546-kube-api-access-kmgk6\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.126601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.126588    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkndh\" (UniqueName: \"kubernetes.io/projected/b3755f62-e630-42ea-85db-fd9a2adc01eb-kube-api-access-dkndh\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.126601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.126596    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3755f62-e630-42ea-85db-fd9a2adc01eb-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:04.154534 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.154510    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:04.568331 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.568299    2555 generic.go:356] "Generic (PLEG): container finished" podID="92be1b3c-1576-4df3-a07f-9dc398358546" containerID="0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063" exitCode=0
Jul 29 01:38:04.568746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.568386    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw" event={"ID":"92be1b3c-1576-4df3-a07f-9dc398358546","Type":"ContainerDied","Data":"0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063"}
Jul 29 01:38:04.568746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.568401    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"
Jul 29 01:38:04.568746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.568472    2555 scope.go:117] "RemoveContainer" containerID="0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063"
Jul 29 01:38:04.568746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.568457    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw" event={"ID":"92be1b3c-1576-4df3-a07f-9dc398358546","Type":"ContainerDied","Data":"18557c6c35a980bfa2bbaaf9c59eae22b0d9fc41c9e71218f274950fbd45f7bb"}
Jul 29 01:38:04.569654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.569628    2555 generic.go:356] "Generic (PLEG): container finished" podID="b3755f62-e630-42ea-85db-fd9a2adc01eb" containerID="3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8" exitCode=0
Jul 29 01:38:04.569773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.569673    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"
Jul 29 01:38:04.569773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.569666    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5" event={"ID":"b3755f62-e630-42ea-85db-fd9a2adc01eb","Type":"ContainerDied","Data":"3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8"}
Jul 29 01:38:04.569773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.569727    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5" event={"ID":"b3755f62-e630-42ea-85db-fd9a2adc01eb","Type":"ContainerDied","Data":"0a33ef8e6b455f387ba78b17f4971b8b8a021a520db27343203733ec4daf03b1"}
Jul 29 01:38:04.571072 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.571049    2555 generic.go:356] "Generic (PLEG): container finished" podID="84ad165d-9e24-4aba-9ff9-b06c8cee02ca" containerID="e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0" exitCode=0
Jul 29 01:38:04.571181 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.571097    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"
Jul 29 01:38:04.571181 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.571104    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc" event={"ID":"84ad165d-9e24-4aba-9ff9-b06c8cee02ca","Type":"ContainerDied","Data":"e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0"}
Jul 29 01:38:04.571181 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.571140    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc" event={"ID":"84ad165d-9e24-4aba-9ff9-b06c8cee02ca","Type":"ContainerDied","Data":"6f25bb530e6d0f9f5739c6ddd61f07442ac40b5a08197f56dbbe9050ba4de232"}
Jul 29 01:38:04.578086 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.577875    2555 scope.go:117] "RemoveContainer" containerID="0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063"
Jul 29 01:38:04.578162 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:38:04.578141    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063\": container with ID starting with 0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063 not found: ID does not exist" containerID="0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063"
Jul 29 01:38:04.578224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.578176    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063"} err="failed to get container status \"0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063\": rpc error: code = NotFound desc = could not find container \"0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063\": container with ID starting with 0e68076e9cf474d8db2b7dc55e376baa992a6e9513b95118c0bf7604af077063 not found: ID does not exist"
Jul 29 01:38:04.578224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.578200    2555 scope.go:117] "RemoveContainer" containerID="3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8"
Jul 29 01:38:04.597572 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.597555    2555 scope.go:117] "RemoveContainer" containerID="3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8"
Jul 29 01:38:04.597881 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:38:04.597856    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8\": container with ID starting with 3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8 not found: ID does not exist" containerID="3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8"
Jul 29 01:38:04.597964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.597890    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8"} err="failed to get container status \"3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8\": rpc error: code = NotFound desc = could not find container \"3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8\": container with ID starting with 3c386a0cabd727bf6b639d78a0ec2f0b90710b49e4d043ad51fcdcd8099683f8 not found: ID does not exist"
Jul 29 01:38:04.597964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.597913    2555 scope.go:117] "RemoveContainer" containerID="e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0"
Jul 29 01:38:04.605346 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.605327    2555 scope.go:117] "RemoveContainer" containerID="e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0"
Jul 29 01:38:04.605586 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:38:04.605568    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0\": container with ID starting with e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0 not found: ID does not exist" containerID="e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0"
Jul 29 01:38:04.605636 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.605592    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0"} err="failed to get container status \"e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0\": rpc error: code = NotFound desc = could not find container \"e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0\": container with ID starting with e7a259e16c7c4d779a7aec47bf8a6c6a9b82e65f0283dee45d9f714de2c7bcc0 not found: ID does not exist"
Jul 29 01:38:04.611068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.611048    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"]
Jul 29 01:38:04.617550 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.617530    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aa4526-9b85b44bd-4lvsc"]
Jul 29 01:38:04.627679 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.627661    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"]
Jul 29 01:38:04.633297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.633261    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-aa4526-764bf865f-sf6vw"]
Jul 29 01:38:04.643764 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.643743    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"]
Jul 29 01:38:04.647516 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:04.647495    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aa4526-56b4766578-jmkk5"]
Jul 29 01:38:05.190915 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:05.190871    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84ad165d-9e24-4aba-9ff9-b06c8cee02ca" path="/var/lib/kubelet/pods/84ad165d-9e24-4aba-9ff9-b06c8cee02ca/volumes"
Jul 29 01:38:05.191397 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:05.191373    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92be1b3c-1576-4df3-a07f-9dc398358546" path="/var/lib/kubelet/pods/92be1b3c-1576-4df3-a07f-9dc398358546/volumes"
Jul 29 01:38:05.191782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:05.191762    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3755f62-e630-42ea-85db-fd9a2adc01eb" path="/var/lib/kubelet/pods/b3755f62-e630-42ea-85db-fd9a2adc01eb/volumes"
Jul 29 01:38:10.191358 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:10.191321    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:10.303104 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:10.303067    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:10.449616 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:10.449525    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"]
Jul 29 01:38:10.449801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:10.449725    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy" containerID="cri-o://77b920ae2c993465be7628d337bea352b81c29170d867962054baef663f11dee" gracePeriod=30
Jul 29 01:38:10.480339 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:10.480304    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:10.669860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:10.669832    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:15.635364 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.635334    2555 generic.go:356] "Generic (PLEG): container finished" podID="fcd6026c-e1af-452c-823d-0077e854deef" containerID="77b920ae2c993465be7628d337bea352b81c29170d867962054baef663f11dee" exitCode=0
Jul 29 01:38:15.635770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.635408    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" event={"ID":"fcd6026c-e1af-452c-823d-0077e854deef","Type":"ContainerDied","Data":"77b920ae2c993465be7628d337bea352b81c29170d867962054baef663f11dee"}
Jul 29 01:38:15.635770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.635482    2555 scope.go:117] "RemoveContainer" containerID="79575e1b15e33fa33d61f67986f9c0fef7d83de282695c7f17480ebeee93062f"
Jul 29 01:38:15.713001 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.712927    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:38:15.839302 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839265    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-socket\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839467 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839306    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-data\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839467 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839365    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd6026c-e1af-452c-823d-0077e854deef-istio-podinfo\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839467 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839427    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-credential-socket\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839467 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839460    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd6026c-e1af-452c-823d-0077e854deef-istiod-ca-cert\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839499    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-istio-token\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839554    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf5j8\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-kube-api-access-kf5j8\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839561    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:15.839690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839578    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-certs\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839690 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839633    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-envoy\") pod \"fcd6026c-e1af-452c-823d-0077e854deef\" (UID: \"fcd6026c-e1af-452c-823d-0077e854deef\") "
Jul 29 01:38:15.839956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839833    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:15.839956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839916    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd6026c-e1af-452c-823d-0077e854deef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:38:15.840073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.839990    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.840073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.840010    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-credential-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.840182 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.840159    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:15.840236 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.840194    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:15.842132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.842104    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcd6026c-e1af-452c-823d-0077e854deef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:38:15.842312 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.842189    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-kube-api-access-kf5j8" (OuterVolumeSpecName: "kube-api-access-kf5j8") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "kube-api-access-kf5j8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:15.842312 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.842235    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:15.842312 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.842259    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcd6026c-e1af-452c-823d-0077e854deef" (UID: "fcd6026c-e1af-452c-823d-0077e854deef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:15.941128 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941099    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf5j8\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-kube-api-access-kf5j8\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.941128 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941124    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-workload-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.941309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941137    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-envoy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.941309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941148    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd6026c-e1af-452c-823d-0077e854deef-istio-data\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.941309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941160    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd6026c-e1af-452c-823d-0077e854deef-istio-podinfo\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.941309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941171    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd6026c-e1af-452c-823d-0077e854deef-istiod-ca-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:15.941309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:15.941182    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd6026c-e1af-452c-823d-0077e854deef-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:16.642679 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:16.642648    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"
Jul 29 01:38:16.643098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:16.642645    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw" event={"ID":"fcd6026c-e1af-452c-823d-0077e854deef","Type":"ContainerDied","Data":"f2d7b5b81465455ebaffdec5e3c02edf9bdd955826cc2a1f038ecb873a1de06d"}
Jul 29 01:38:16.643098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:16.642782    2555 scope.go:117] "RemoveContainer" containerID="77b920ae2c993465be7628d337bea352b81c29170d867962054baef663f11dee"
Jul 29 01:38:16.678702 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:16.678680    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"]
Jul 29 01:38:16.692672 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:16.692649    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-aa4526-openshift-default-df5567cc4-qk2bw"]
Jul 29 01:38:17.190250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:17.190214    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcd6026c-e1af-452c-823d-0077e854deef" path="/var/lib/kubelet/pods/fcd6026c-e1af-452c-823d-0077e854deef/volumes"
Jul 29 01:38:24.943760 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:24.943716    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:26.254991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.254956    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:26.289382 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.289345    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"]
Jul 29 01:38:26.289657 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.289632    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c" podUID="7b7d6f47-ab4d-474c-b82d-6207c73a41f4" containerName="payload-processing" containerID="cri-o://7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1" gracePeriod=30
Jul 29 01:38:26.356630 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.356587    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:26.436605 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.436566    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:26.502489 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.502456    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:26.550114 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.550091    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:38:26.641341 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.641304    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-plugins-config-volume\") pod \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") "
Jul 29 01:38:26.641495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.641387    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z5vp\" (UniqueName: \"kubernetes.io/projected/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-kube-api-access-6z5vp\") pod \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\" (UID: \"7b7d6f47-ab4d-474c-b82d-6207c73a41f4\") "
Jul 29 01:38:26.641694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.641671    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b7d6f47-ab4d-474c-b82d-6207c73a41f4" (UID: "7b7d6f47-ab4d-474c-b82d-6207c73a41f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:38:26.643409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.643385    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-kube-api-access-6z5vp" (OuterVolumeSpecName: "kube-api-access-6z5vp") pod "7b7d6f47-ab4d-474c-b82d-6207c73a41f4" (UID: "7b7d6f47-ab4d-474c-b82d-6207c73a41f4"). InnerVolumeSpecName "kube-api-access-6z5vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:26.698306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.698258    2555 generic.go:356] "Generic (PLEG): container finished" podID="7b7d6f47-ab4d-474c-b82d-6207c73a41f4" containerID="7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1" exitCode=0
Jul 29 01:38:26.698497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.698332    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"
Jul 29 01:38:26.698497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.698346    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c" event={"ID":"7b7d6f47-ab4d-474c-b82d-6207c73a41f4","Type":"ContainerDied","Data":"7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1"}
Jul 29 01:38:26.698497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.698411    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c" event={"ID":"7b7d6f47-ab4d-474c-b82d-6207c73a41f4","Type":"ContainerDied","Data":"2ce2a73132eb529e3b7a2ee6a73b839312b899cbf18b8b2f03aa260b90bc5bde"}
Jul 29 01:38:26.698497 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.698441    2555 scope.go:117] "RemoveContainer" containerID="7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1"
Jul 29 01:38:26.708300 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.708108    2555 scope.go:117] "RemoveContainer" containerID="7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1"
Jul 29 01:38:26.708544 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:38:26.708503    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1\": container with ID starting with 7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1 not found: ID does not exist" containerID="7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1"
Jul 29 01:38:26.708673 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.708557    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1"} err="failed to get container status \"7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1\": rpc error: code = NotFound desc = could not find container \"7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1\": container with ID starting with 7d8eda806b11841bb63a51754054ab2df2a4e04ff5f2d99e513ecd28572a45b1 not found: ID does not exist"
Jul 29 01:38:26.722073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.722044    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"]
Jul 29 01:38:26.725440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.725420    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4126fe-674b5f8cc8-9bm8c"]
Jul 29 01:38:26.742517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.742487    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:26.742517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.742511    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z5vp\" (UniqueName: \"kubernetes.io/projected/7b7d6f47-ab4d-474c-b82d-6207c73a41f4-kube-api-access-6z5vp\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:26.814641 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:26.814552    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:27.190681 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:27.190652    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7d6f47-ab4d-474c-b82d-6207c73a41f4" path="/var/lib/kubelet/pods/7b7d6f47-ab4d-474c-b82d-6207c73a41f4/volumes"
Jul 29 01:38:32.463796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:32.463762    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:32.563736 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:32.563706    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:32.706420 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:32.706389    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"]
Jul 29 01:38:32.706671 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:32.706639    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy" containerID="cri-o://455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e" gracePeriod=30
Jul 29 01:38:32.724990 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:32.724918    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:32.944526 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:32.944490    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:33.284387 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:33.284350    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:33.597041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:33.596960    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:33.836494 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:33.836463    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:36.324016 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:36.323953    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:36.424170 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:36.424129    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:36.553321 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:36.553261    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:36.895476 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:36.895441    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:37.246074 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.246037    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:37.343586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.343552    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:38:37.376677 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.376650    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"]
Jul 29 01:38:37.377111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377074    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84ad165d-9e24-4aba-9ff9-b06c8cee02ca" containerName="payload-pre-processing"
Jul 29 01:38:37.377111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377100    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="84ad165d-9e24-4aba-9ff9-b06c8cee02ca" containerName="payload-pre-processing"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377124    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92be1b3c-1576-4df3-a07f-9dc398358546" containerName="maas-api"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377133    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="92be1b3c-1576-4df3-a07f-9dc398358546" containerName="maas-api"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377169    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377178    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377194    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7d6f47-ab4d-474c-b82d-6207c73a41f4" containerName="payload-processing"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377203    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7d6f47-ab4d-474c-b82d-6207c73a41f4" containerName="payload-processing"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377223    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3755f62-e630-42ea-85db-fd9a2adc01eb" containerName="payload-processing"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377231    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3755f62-e630-42ea-85db-fd9a2adc01eb" containerName="payload-processing"
Jul 29 01:38:37.377383 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377375    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy"
Jul 29 01:38:37.377872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377394    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7d6f47-ab4d-474c-b82d-6207c73a41f4" containerName="payload-processing"
Jul 29 01:38:37.377872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377407    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="84ad165d-9e24-4aba-9ff9-b06c8cee02ca" containerName="payload-pre-processing"
Jul 29 01:38:37.377872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377442    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3755f62-e630-42ea-85db-fd9a2adc01eb" containerName="payload-processing"
Jul 29 01:38:37.377872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.377451    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="92be1b3c-1576-4df3-a07f-9dc398358546" containerName="maas-api"
Jul 29 01:38:37.382226 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.382208    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.385525 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.385503    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-5ea1e0-dockercfg-pxc52\""
Jul 29 01:38:37.385653 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.385542    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-5ea1e0\""
Jul 29 01:38:37.391561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.391537    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"]
Jul 29 01:38:37.425382 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.425354    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z72wm\" (UniqueName: \"kubernetes.io/projected/35ddf2a8-f8f6-4767-8903-2aa696b091db-kube-api-access-z72wm\") pod \"payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.425485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.425397    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ddf2a8-f8f6-4767-8903-2aa696b091db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.526969 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.526877    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ddf2a8-f8f6-4767-8903-2aa696b091db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.527149 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.527002    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z72wm\" (UniqueName: \"kubernetes.io/projected/35ddf2a8-f8f6-4767-8903-2aa696b091db-kube-api-access-z72wm\") pod \"payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.527847 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.527821    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ddf2a8-f8f6-4767-8903-2aa696b091db-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.539126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.539096    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z72wm\" (UniqueName: \"kubernetes.io/projected/35ddf2a8-f8f6-4767-8903-2aa696b091db-kube-api-access-z72wm\") pod \"payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") " pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.694107 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.694074    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:37.841221 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.841197    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"]
Jul 29 01:38:37.843197 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:38:37.843167    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ddf2a8_f8f6_4767_8903_2aa696b091db.slice/crio-741165a5293a4d5be8c7faa9bad576603033a21c28e47ede80bd59b92d66ce98 WatchSource:0}: Error finding container 741165a5293a4d5be8c7faa9bad576603033a21c28e47ede80bd59b92d66ce98: Status 404 returned error can't find the container with id 741165a5293a4d5be8c7faa9bad576603033a21c28e47ede80bd59b92d66ce98
Jul 29 01:38:37.954803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:37.954774    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:38:38.032391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.032328    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-credential-socket\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.032784 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.032707    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1950c97-7a58-4c42-9176-d779f6dad1eb-istiod-ca-cert\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.032948    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-token\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.032993    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-podinfo\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.033297    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-data\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.033483    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-certs\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.033542    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z26dn\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-kube-api-access-z26dn\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.033794    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-envoy\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.033954 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.033832    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-socket\") pod \"b1950c97-7a58-4c42-9176-d779f6dad1eb\" (UID: \"b1950c97-7a58-4c42-9176-d779f6dad1eb\") "
Jul 29 01:38:38.034527 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.034457    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:38.034866 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.034787    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:38.034988 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.034948    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:38.035096 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.035003    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1950c97-7a58-4c42-9176-d779f6dad1eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:38:38.035096 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.035084    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:38.037390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.037303    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-kube-api-access-z26dn" (OuterVolumeSpecName: "kube-api-access-z26dn") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "kube-api-access-z26dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:38.037390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.037350    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:38:38.037876 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.037857    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:38:38.037968 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.037915    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1950c97-7a58-4c42-9176-d779f6dad1eb" (UID: "b1950c97-7a58-4c42-9176-d779f6dad1eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:38:38.135033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.134997    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-data\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135033 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135032    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135048    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z26dn\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-kube-api-access-z26dn\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135062    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-envoy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135075    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-workload-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135088    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1950c97-7a58-4c42-9176-d779f6dad1eb-credential-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135100    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1950c97-7a58-4c42-9176-d779f6dad1eb-istiod-ca-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135113    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.135258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.135124    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1950c97-7a58-4c42-9176-d779f6dad1eb-istio-podinfo\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:38:38.766250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.766214    2555 generic.go:356] "Generic (PLEG): container finished" podID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerID="455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e" exitCode=0
Jul 29 01:38:38.766742 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.766258    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"
Jul 29 01:38:38.766742 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.766297    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" event={"ID":"b1950c97-7a58-4c42-9176-d779f6dad1eb","Type":"ContainerDied","Data":"455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e"}
Jul 29 01:38:38.766742 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.766375    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks" event={"ID":"b1950c97-7a58-4c42-9176-d779f6dad1eb","Type":"ContainerDied","Data":"12245d9921bb90e4c797bc2087c3c7f67f3295e07598d2b1055ba4fb3baadb11"}
Jul 29 01:38:38.766742 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.766411    2555 scope.go:117] "RemoveContainer" containerID="455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e"
Jul 29 01:38:38.768178 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.768142    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff" event={"ID":"35ddf2a8-f8f6-4767-8903-2aa696b091db","Type":"ContainerStarted","Data":"fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161"}
Jul 29 01:38:38.768297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.768199    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:38:38.768297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.768222    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff" event={"ID":"35ddf2a8-f8f6-4767-8903-2aa696b091db","Type":"ContainerStarted","Data":"741165a5293a4d5be8c7faa9bad576603033a21c28e47ede80bd59b92d66ce98"}
Jul 29 01:38:38.776503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.776487    2555 scope.go:117] "RemoveContainer" containerID="f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"
Jul 29 01:38:38.784208 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.784194    2555 scope.go:117] "RemoveContainer" containerID="455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e"
Jul 29 01:38:38.784490 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:38:38.784474    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e\": container with ID starting with 455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e not found: ID does not exist" containerID="455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e"
Jul 29 01:38:38.784535 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.784498    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e"} err="failed to get container status \"455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e\": rpc error: code = NotFound desc = could not find container \"455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e\": container with ID starting with 455a6e8eff3bc5f4af603f52ef9ab835ae095efba8eb22978f6da7df8c90ed5e not found: ID does not exist"
Jul 29 01:38:38.784535 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.784513    2555 scope.go:117] "RemoveContainer" containerID="f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"
Jul 29 01:38:38.784730 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:38:38.784712    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf\": container with ID starting with f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf not found: ID does not exist" containerID="f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"
Jul 29 01:38:38.784765 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.784739    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf"} err="failed to get container status \"f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf\": rpc error: code = NotFound desc = could not find container \"f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf\": container with ID starting with f9c3eb83b8491b2fce106dd339ef2327a0f788218f4e15f37f06ab4974382baf not found: ID does not exist"
Jul 29 01:38:38.788309 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.788251    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff" podStartSLOduration=1.788238934 podStartE2EDuration="1.788238934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:38:38.787247401 +0000 UTC m=+2648.117897006" watchObservedRunningTime="2026-07-29 01:38:38.788238934 +0000 UTC m=+2648.118888538"
Jul 29 01:38:38.803741 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.803693    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"]
Jul 29 01:38:38.806803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:38.806781    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4126fe-openshift-default-7bd78b7758-fkkks"]
Jul 29 01:38:39.189214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:39.189188    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" path="/var/lib/kubelet/pods/b1950c97-7a58-4c42-9176-d779f6dad1eb/volumes"
Jul 29 01:38:49.776699 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:38:49.776668    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:39:01.301585 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.301549    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:01.874090 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874048    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"]
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874677    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874704    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874731    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874739    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874751    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874761    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874850    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874862    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1950c97-7a58-4c42-9176-d779f6dad1eb" containerName="istio-proxy"
Jul 29 01:39:01.874940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.874875    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd6026c-e1af-452c-823d-0077e854deef" containerName="istio-proxy"
Jul 29 01:39:01.879974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.879954    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:01.882647 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.882624    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 01:39:01.882767 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.882633    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-da474e-openshift-default-dockercfg-gmfxn\""
Jul 29 01:39:01.891378 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:01.891358    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"]
Jul 29 01:39:02.059956 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.059923    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-envoy\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.059965    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-podinfo\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.059986    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-credential-socket\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.060042    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4ff32af-ea91-46eb-a954-a3f1397e1371-istiod-ca-cert\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.060092    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvwrn\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-kube-api-access-bvwrn\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060286 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.060127    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-data\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060286 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.060149    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-certs\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060286 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.060178    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-socket\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.060286 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.060194    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-token\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160712 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160621    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4ff32af-ea91-46eb-a954-a3f1397e1371-istiod-ca-cert\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160712 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160677    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvwrn\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-kube-api-access-bvwrn\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160713    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-data\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160737    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-certs\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160768    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-socket\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160790    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-token\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160875    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-envoy\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.160941 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160916    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-podinfo\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.161258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.160943    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-credential-socket\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.161258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.161112    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-certs\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.161258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.161171    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-socket\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.161258 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.161223    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-data\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.161448 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.161314    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-credential-socket\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.161448 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.161423    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4ff32af-ea91-46eb-a954-a3f1397e1371-istiod-ca-cert\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.163081 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.163050    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-envoy\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.163466 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.163446    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-podinfo\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.172751 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.172721    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvwrn\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-kube-api-access-bvwrn\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.172862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.172801    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-token\") pod \"e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") " pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.191437 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.191414    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:02.336029 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.335998    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"]
Jul 29 01:39:02.339222 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:39:02.339186    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4ff32af_ea91_46eb_a954_a3f1397e1371.slice/crio-406c328856671d2bd045c5da556d8927f8c06eb84503b5d909d3ae6e893d7c9e WatchSource:0}: Error finding container 406c328856671d2bd045c5da556d8927f8c06eb84503b5d909d3ae6e893d7c9e: Status 404 returned error can't find the container with id 406c328856671d2bd045c5da556d8927f8c06eb84503b5d909d3ae6e893d7c9e
Jul 29 01:39:02.341499 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.341467    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:39:02.341579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.341533    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:39:02.341579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.341562    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:39:02.888674 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.888633    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" event={"ID":"a4ff32af-ea91-46eb-a954-a3f1397e1371","Type":"ContainerStarted","Data":"e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"}
Jul 29 01:39:02.888864 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.888695    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" event={"ID":"a4ff32af-ea91-46eb-a954-a3f1397e1371","Type":"ContainerStarted","Data":"406c328856671d2bd045c5da556d8927f8c06eb84503b5d909d3ae6e893d7c9e"}
Jul 29 01:39:02.911960 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.911902    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podStartSLOduration=1.911886928 podStartE2EDuration="1.911886928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:39:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:39:02.90972014 +0000 UTC m=+2672.240369746" watchObservedRunningTime="2026-07-29 01:39:02.911886928 +0000 UTC m=+2672.242536533"
Jul 29 01:39:02.922347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:02.922310    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:03.192110 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:03.192080    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:03.197147 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:03.197121    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:03.894257 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:03.894223    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:03.895334 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:03.895317    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:04.805405 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:04.805159    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:04.898623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:04.898587    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:05.007747 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.007715    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"]
Jul 29 01:39:05.011697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.011677    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.014403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.014382    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-da474e-serving-cert\""
Jul 29 01:39:05.022703 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.022678    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"]
Jul 29 01:39:05.040026 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.040002    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:05.188814 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.188786    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12b7d16-a24a-4e0f-9b0b-1117f887c677-maas-api-tls\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.188975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.188851    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdr7\" (UniqueName: \"kubernetes.io/projected/c12b7d16-a24a-4e0f-9b0b-1117f887c677-kube-api-access-ftdr7\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.188975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.188950    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12b7d16-a24a-4e0f-9b0b-1117f887c677-openshift-service-ca\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.261136 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.261081    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:05.290020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.289992    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdr7\" (UniqueName: \"kubernetes.io/projected/c12b7d16-a24a-4e0f-9b0b-1117f887c677-kube-api-access-ftdr7\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.290207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.290043    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12b7d16-a24a-4e0f-9b0b-1117f887c677-openshift-service-ca\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.290207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.290093    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12b7d16-a24a-4e0f-9b0b-1117f887c677-maas-api-tls\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.290676 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.290655    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12b7d16-a24a-4e0f-9b0b-1117f887c677-openshift-service-ca\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.292459 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.292439    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12b7d16-a24a-4e0f-9b0b-1117f887c677-maas-api-tls\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.298094 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.298070    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdr7\" (UniqueName: \"kubernetes.io/projected/c12b7d16-a24a-4e0f-9b0b-1117f887c677-kube-api-access-ftdr7\") pod \"maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.324268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.324246    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:05.459796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.459769    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"]
Jul 29 01:39:05.461511 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:39:05.461469    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12b7d16_a24a_4e0f_9b0b_1117f887c677.slice/crio-4e000b74784fa86f25755e9ae292d98a8957af547045e8424e0c1047c1b0799d WatchSource:0}: Error finding container 4e000b74784fa86f25755e9ae292d98a8957af547045e8424e0c1047c1b0799d: Status 404 returned error can't find the container with id 4e000b74784fa86f25755e9ae292d98a8957af547045e8424e0c1047c1b0799d
Jul 29 01:39:05.643450 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.643420    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:05.756856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.756764    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:05.780092 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.780061    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"]
Jul 29 01:39:05.784955 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.784934    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:05.787779 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.787758    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-da474e\""
Jul 29 01:39:05.787949 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.787933    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-da474e-dockercfg-k2q2h\""
Jul 29 01:39:05.799565 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.799539    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"]
Jul 29 01:39:05.895020 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.894990    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhl6l\" (UniqueName: \"kubernetes.io/projected/723be36f-4045-4762-8571-756bbc2d5c0f-kube-api-access-dhl6l\") pod \"payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:05.895150 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.895077    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723be36f-4045-4762-8571-756bbc2d5c0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:05.906332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.906306    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr" event={"ID":"c12b7d16-a24a-4e0f-9b0b-1117f887c677","Type":"ContainerStarted","Data":"4e000b74784fa86f25755e9ae292d98a8957af547045e8424e0c1047c1b0799d"}
Jul 29 01:39:05.995765 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.995731    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723be36f-4045-4762-8571-756bbc2d5c0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:05.995957 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.995846    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhl6l\" (UniqueName: \"kubernetes.io/projected/723be36f-4045-4762-8571-756bbc2d5c0f-kube-api-access-dhl6l\") pod \"payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:05.996513 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:05.996485    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723be36f-4045-4762-8571-756bbc2d5c0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:06.004442 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.004418    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhl6l\" (UniqueName: \"kubernetes.io/projected/723be36f-4045-4762-8571-756bbc2d5c0f-kube-api-access-dhl6l\") pod \"payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:06.097469 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.097370    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:06.240132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.240103    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"]
Jul 29 01:39:06.240623 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:39:06.240602    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723be36f_4045_4762_8571_756bbc2d5c0f.slice/crio-1a785eaa0dc8bd171c287ebea4a72d05aab799732489dc4c7c43541913e2ecde WatchSource:0}: Error finding container 1a785eaa0dc8bd171c287ebea4a72d05aab799732489dc4c7c43541913e2ecde: Status 404 returned error can't find the container with id 1a785eaa0dc8bd171c287ebea4a72d05aab799732489dc4c7c43541913e2ecde
Jul 29 01:39:06.915327 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.915267    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr" event={"ID":"c12b7d16-a24a-4e0f-9b0b-1117f887c677","Type":"ContainerStarted","Data":"21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c"}
Jul 29 01:39:06.915791 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.915657    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:06.917851 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.917822    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42" event={"ID":"723be36f-4045-4762-8571-756bbc2d5c0f","Type":"ContainerStarted","Data":"fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c"}
Jul 29 01:39:06.917991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.917868    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42" event={"ID":"723be36f-4045-4762-8571-756bbc2d5c0f","Type":"ContainerStarted","Data":"1a785eaa0dc8bd171c287ebea4a72d05aab799732489dc4c7c43541913e2ecde"}
Jul 29 01:39:06.917991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.917905    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:06.935548 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.935486    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr" podStartSLOduration=2.530366774 podStartE2EDuration="2.935472265s" totalImagesPullingTime="405.105491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:39:06.932415911 +0000 UTC m=+2676.263065515" watchObservedRunningTime="2026-07-29 01:39:06.935472265 +0000 UTC m=+2676.266121861"
Jul 29 01:39:06.950838 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:06.950786    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42" podStartSLOduration=1.950768778 podStartE2EDuration="1.950768778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:39:06.949227339 +0000 UTC m=+2676.279876946" watchObservedRunningTime="2026-07-29 01:39:06.950768778 +0000 UTC m=+2676.281418384"
Jul 29 01:39:12.929063 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:12.929024    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:39:17.925651 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:17.925618    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:39:19.288197 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:19.288148    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:19.936454 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:19.936413    2555 patch_prober.go:28] interesting pod/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 01:39:19.936621 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:19.936490    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 01:39:21.999586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:21.999551    2555 generic.go:356] "Generic (PLEG): container finished" podID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerID="e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d" exitCode=0
Jul 29 01:39:22.000031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:21.999627    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" event={"ID":"a4ff32af-ea91-46eb-a954-a3f1397e1371","Type":"ContainerDied","Data":"e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"}
Jul 29 01:39:22.000031 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:22.000026    2555 scope.go:117] "RemoveContainer" containerID="e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"
Jul 29 01:39:22.001029 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:22.001005    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:39:22.001083 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:22.001068    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:39:22.001130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:22.001098    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:39:23.006143 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:23.006066    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" event={"ID":"a4ff32af-ea91-46eb-a954-a3f1397e1371","Type":"ContainerStarted","Data":"50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070"}
Jul 29 01:39:23.191878 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:23.191846    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:23.192141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:23.192118    2555 patch_prober.go:28] interesting pod/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 29 01:39:23.192239 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:23.192171    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 29 01:39:24.192599 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:24.192566    2555 patch_prober.go:28] interesting pod/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 29 01:39:24.193052 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:24.192621    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 29 01:39:25.192599 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:25.192565    2555 patch_prober.go:28] interesting pod/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 29 01:39:25.192986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:25.192615    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 29 01:39:26.192072 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:26.192036    2555 patch_prober.go:28] interesting pod/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Jul 29 01:39:26.192251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:26.192090    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Jul 29 01:39:27.196014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:27.195983    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:27.196467 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:27.196231    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:27.196964 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:27.196947    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:39:29.777673 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:29.777645    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:39:35.897795 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:39:35.897750    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:40:01.594721 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.594686    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"]
Jul 29 01:40:01.604000 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.603970    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.605942 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.605914    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"]
Jul 29 01:40:01.608947 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.608922    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"default-dockercfg-vhcjn\""
Jul 29 01:40:01.609219 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.609200    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"auth-test-model-da474e-kserve-self-signed-certs\""
Jul 29 01:40:01.610110 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.609445    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"openshift-service-ca.crt\""
Jul 29 01:40:01.611641 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.611365    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"kube-root-ca.crt\""
Jul 29 01:40:01.698984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.698947    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6053e346-9e32-43b1-8535-3053d1bd4352-tls-certs\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.699192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.698994    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-home\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.699192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.699066    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-model-cache\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.699192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.699162    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-dshm\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.699394 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.699231    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6slh\" (UniqueName: \"kubernetes.io/projected/6053e346-9e32-43b1-8535-3053d1bd4352-kube-api-access-r6slh\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.699394 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.699268    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-tmp-dir\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.781837 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.781798    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:40:01.799991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.799960    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-dshm\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800014    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6slh\" (UniqueName: \"kubernetes.io/projected/6053e346-9e32-43b1-8535-3053d1bd4352-kube-api-access-r6slh\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800037    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-tmp-dir\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800067    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6053e346-9e32-43b1-8535-3053d1bd4352-tls-certs\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800086    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-home\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800139 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800108    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-model-cache\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800494 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800469    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-tmp-dir\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800549 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800507    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-model-cache\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.800590 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.800542    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-home\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.802148 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.802128    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-dshm\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.802434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.802417    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6053e346-9e32-43b1-8535-3053d1bd4352-tls-certs\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.808079 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.808057    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6slh\" (UniqueName: \"kubernetes.io/projected/6053e346-9e32-43b1-8535-3053d1bd4352-kube-api-access-r6slh\") pod \"auth-test-model-da474e-kserve-75fb74fc47-ct5vc\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") " pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:01.918773 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:01.918690    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:02.059728 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:02.059702    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"]
Jul 29 01:40:02.062008 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:40:02.061968    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6053e346_9e32_43b1_8535_3053d1bd4352.slice/crio-aa01e33a8df58acdc806c69bc41501a5bdf89ef78b8727f7e15e0090aefe2125 WatchSource:0}: Error finding container aa01e33a8df58acdc806c69bc41501a5bdf89ef78b8727f7e15e0090aefe2125: Status 404 returned error can't find the container with id aa01e33a8df58acdc806c69bc41501a5bdf89ef78b8727f7e15e0090aefe2125
Jul 29 01:40:02.206008 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:02.205970    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc" event={"ID":"6053e346-9e32-43b1-8535-3053d1bd4352","Type":"ContainerStarted","Data":"dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e"}
Jul 29 01:40:02.206008 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:02.206010    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc" event={"ID":"6053e346-9e32-43b1-8535-3053d1bd4352","Type":"ContainerStarted","Data":"aa01e33a8df58acdc806c69bc41501a5bdf89ef78b8727f7e15e0090aefe2125"}
Jul 29 01:40:02.226255 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:02.226206    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc" podStartSLOduration=1.226189833 podStartE2EDuration="1.226189833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:40:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:40:02.223798603 +0000 UTC m=+2731.554448233" watchObservedRunningTime="2026-07-29 01:40:02.226189833 +0000 UTC m=+2731.556839437"
Jul 29 01:40:03.938991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:03.938959    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:40:11.919271 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:11.919237    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:11.919728 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:11.919304    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:11.931716 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:11.931683    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:40:12.267883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:40:12.267855    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:41:53.855959 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:41:53.855911    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:44:35.951632 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:35.951595    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:44:36.801104 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:36.801069    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"]
Jul 29 01:44:37.158426 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:37.158347    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:44:51.820457 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:51.820396    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc" podUID="6053e346-9e32-43b1-8535-3053d1bd4352" containerName="main" containerID="cri-o://dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e" gracePeriod=45
Jul 29 01:44:52.083512 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.083448    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:44:52.214817 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.214773    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-dshm\") pod \"6053e346-9e32-43b1-8535-3053d1bd4352\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") "
Jul 29 01:44:52.215076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.214888    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6053e346-9e32-43b1-8535-3053d1bd4352-tls-certs\") pod \"6053e346-9e32-43b1-8535-3053d1bd4352\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") "
Jul 29 01:44:52.215076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.214945    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-model-cache\") pod \"6053e346-9e32-43b1-8535-3053d1bd4352\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") "
Jul 29 01:44:52.215076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.214976    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6slh\" (UniqueName: \"kubernetes.io/projected/6053e346-9e32-43b1-8535-3053d1bd4352-kube-api-access-r6slh\") pod \"6053e346-9e32-43b1-8535-3053d1bd4352\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") "
Jul 29 01:44:52.215076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.215030    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-tmp-dir\") pod \"6053e346-9e32-43b1-8535-3053d1bd4352\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") "
Jul 29 01:44:52.215076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.215072    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-home\") pod \"6053e346-9e32-43b1-8535-3053d1bd4352\" (UID: \"6053e346-9e32-43b1-8535-3053d1bd4352\") "
Jul 29 01:44:52.215390 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.215298    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-model-cache" (OuterVolumeSpecName: "model-cache") pod "6053e346-9e32-43b1-8535-3053d1bd4352" (UID: "6053e346-9e32-43b1-8535-3053d1bd4352"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:52.215457 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.215394    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6053e346-9e32-43b1-8535-3053d1bd4352" (UID: "6053e346-9e32-43b1-8535-3053d1bd4352"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:52.215457 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.215424    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-home" (OuterVolumeSpecName: "home") pod "6053e346-9e32-43b1-8535-3053d1bd4352" (UID: "6053e346-9e32-43b1-8535-3053d1bd4352"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:52.215559 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.215454    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-model-cache\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:44:52.217131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.217101    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-dshm" (OuterVolumeSpecName: "dshm") pod "6053e346-9e32-43b1-8535-3053d1bd4352" (UID: "6053e346-9e32-43b1-8535-3053d1bd4352"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:44:52.217229 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.217160    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6053e346-9e32-43b1-8535-3053d1bd4352-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6053e346-9e32-43b1-8535-3053d1bd4352" (UID: "6053e346-9e32-43b1-8535-3053d1bd4352"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:44:52.217316 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.217257    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6053e346-9e32-43b1-8535-3053d1bd4352-kube-api-access-r6slh" (OuterVolumeSpecName: "kube-api-access-r6slh") pod "6053e346-9e32-43b1-8535-3053d1bd4352" (UID: "6053e346-9e32-43b1-8535-3053d1bd4352"). InnerVolumeSpecName "kube-api-access-r6slh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:44:52.316886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.316843    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6slh\" (UniqueName: \"kubernetes.io/projected/6053e346-9e32-43b1-8535-3053d1bd4352-kube-api-access-r6slh\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:44:52.316886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.316881    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-tmp-dir\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:44:52.316886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.316890    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-home\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:44:52.316886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.316898    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6053e346-9e32-43b1-8535-3053d1bd4352-dshm\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:44:52.317171 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.316906    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6053e346-9e32-43b1-8535-3053d1bd4352-tls-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:44:52.698801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.698764    2555 generic.go:356] "Generic (PLEG): container finished" podID="6053e346-9e32-43b1-8535-3053d1bd4352" containerID="dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e" exitCode=0
Jul 29 01:44:52.698962 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.698829    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"
Jul 29 01:44:52.698962 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.698844    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc" event={"ID":"6053e346-9e32-43b1-8535-3053d1bd4352","Type":"ContainerDied","Data":"dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e"}
Jul 29 01:44:52.698962 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.698891    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc" event={"ID":"6053e346-9e32-43b1-8535-3053d1bd4352","Type":"ContainerDied","Data":"aa01e33a8df58acdc806c69bc41501a5bdf89ef78b8727f7e15e0090aefe2125"}
Jul 29 01:44:52.698962 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.698908    2555 scope.go:117] "RemoveContainer" containerID="dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e"
Jul 29 01:44:52.708338 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.708321    2555 scope.go:117] "RemoveContainer" containerID="dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e"
Jul 29 01:44:52.708562 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:44:52.708543    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e\": container with ID starting with dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e not found: ID does not exist" containerID="dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e"
Jul 29 01:44:52.708622 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.708569    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e"} err="failed to get container status \"dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e\": rpc error: code = NotFound desc = could not find container \"dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e\": container with ID starting with dc0775ce1c767c616385ab2c6bdc59348481106234127341e713e09fb893724e not found: ID does not exist"
Jul 29 01:44:52.722356 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.722334    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"]
Jul 29 01:44:52.724214 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:52.724195    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/auth-test-model-da474e-kserve-75fb74fc47-ct5vc"]
Jul 29 01:44:53.191583 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:53.191551    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6053e346-9e32-43b1-8535-3053d1bd4352" path="/var/lib/kubelet/pods/6053e346-9e32-43b1-8535-3053d1bd4352/volumes"
Jul 29 01:44:53.382644 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:44:53.382610    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:45:00.134580 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.134540    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"]
Jul 29 01:45:00.135251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.135232    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6053e346-9e32-43b1-8535-3053d1bd4352" containerName="main"
Jul 29 01:45:00.135327 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.135256    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6053e346-9e32-43b1-8535-3053d1bd4352" containerName="main"
Jul 29 01:45:00.135408 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.135394    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6053e346-9e32-43b1-8535-3053d1bd4352" containerName="main"
Jul 29 01:45:00.140779 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.140762    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"
Jul 29 01:45:00.143485 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.143467    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-gjvns\""
Jul 29 01:45:00.148208 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.148002    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"]
Jul 29 01:45:00.154856 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.154831    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"
Jul 29 01:45:00.295600 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.295565    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"]
Jul 29 01:45:00.296895 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:45:00.296867    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12296d79_96a8_4a7c_b1f9_c445dd31a952.slice/crio-6e8ce8cd3c8218f2aa541d8217427c6b87cd3270db8fe44d8799830192692329 WatchSource:0}: Error finding container 6e8ce8cd3c8218f2aa541d8217427c6b87cd3270db8fe44d8799830192692329: Status 404 returned error can't find the container with id 6e8ce8cd3c8218f2aa541d8217427c6b87cd3270db8fe44d8799830192692329
Jul 29 01:45:00.743969 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.743931    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj" event={"ID":"12296d79-96a8-4a7c-b1f9-c445dd31a952","Type":"ContainerStarted","Data":"4f2c14f1e25fba12464d8d7682ea5e49061f65645bcef8ef781d0b6cd3ba9c9b"}
Jul 29 01:45:00.744155 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.743992    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj" event={"ID":"12296d79-96a8-4a7c-b1f9-c445dd31a952","Type":"ContainerStarted","Data":"6e8ce8cd3c8218f2aa541d8217427c6b87cd3270db8fe44d8799830192692329"}
Jul 29 01:45:00.761178 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:00.761129    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj" podStartSLOduration=0.761109219 podStartE2EDuration="761.109219ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:45:00.758071445 +0000 UTC m=+3030.088721048" watchObservedRunningTime="2026-07-29 01:45:00.761109219 +0000 UTC m=+3030.091758826"
Jul 29 01:45:01.750584 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:01.750548    2555 generic.go:356] "Generic (PLEG): container finished" podID="12296d79-96a8-4a7c-b1f9-c445dd31a952" containerID="4f2c14f1e25fba12464d8d7682ea5e49061f65645bcef8ef781d0b6cd3ba9c9b" exitCode=0
Jul 29 01:45:01.750975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:01.750602    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj" event={"ID":"12296d79-96a8-4a7c-b1f9-c445dd31a952","Type":"ContainerDied","Data":"4f2c14f1e25fba12464d8d7682ea5e49061f65645bcef8ef781d0b6cd3ba9c9b"}
Jul 29 01:45:02.894696 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:02.894669    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"
Jul 29 01:45:03.763405 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:03.763372    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"
Jul 29 01:45:03.763405 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:03.763388    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj" event={"ID":"12296d79-96a8-4a7c-b1f9-c445dd31a952","Type":"ContainerDied","Data":"6e8ce8cd3c8218f2aa541d8217427c6b87cd3270db8fe44d8799830192692329"}
Jul 29 01:45:03.763620 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:03.763436    2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e8ce8cd3c8218f2aa541d8217427c6b87cd3270db8fe44d8799830192692329"
Jul 29 01:45:24.417719 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:45:24.417624    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:28.734302 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:28.734200    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:29.901802 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:29.901766    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:30.123811 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:30.123777    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:30.224776 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:30.224737    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:33.820649 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:33.820612    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:36.158626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:36.158587    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:51.294534 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.294504    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"]
Jul 29 01:46:51.294933 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.294919    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12296d79-96a8-4a7c-b1f9-c445dd31a952" containerName="cleanup"
Jul 29 01:46:51.294983 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.294943    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="12296d79-96a8-4a7c-b1f9-c445dd31a952" containerName="cleanup"
Jul 29 01:46:51.295022 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.295009    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="12296d79-96a8-4a7c-b1f9-c445dd31a952" containerName="cleanup"
Jul 29 01:46:51.298064 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.298047    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.302068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.302044    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"rate-test-model-da474e-kserve-self-signed-certs\""
Jul 29 01:46:51.302068 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.302059    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"openshift-service-ca.crt\""
Jul 29 01:46:51.302307 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.302120    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"default-dockercfg-vhcjn\""
Jul 29 01:46:51.302307 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.302289    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-da474e\"/\"kube-root-ca.crt\""
Jul 29 01:46:51.314949 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.314925    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"]
Jul 29 01:46:51.316697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.316677    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:46:51.427950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.427912    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f067560b-975b-4922-acaa-4b06d0c69e05-tls-certs\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.427950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.427951    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-dshm\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.428157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.428019    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5wbq\" (UniqueName: \"kubernetes.io/projected/f067560b-975b-4922-acaa-4b06d0c69e05-kube-api-access-w5wbq\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.428157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.428075    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-home\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.428157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.428102    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-tmp-dir\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.428157 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.428141    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-model-cache\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529240    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-home\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529319    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-tmp-dir\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529357    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-model-cache\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529423    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f067560b-975b-4922-acaa-4b06d0c69e05-tls-certs\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529503 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529449    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-dshm\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529718 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529517    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5wbq\" (UniqueName: \"kubernetes.io/projected/f067560b-975b-4922-acaa-4b06d0c69e05-kube-api-access-w5wbq\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529775 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529732    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-home\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.529775 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.529745    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-tmp-dir\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.530120 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.530086    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-model-cache\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.532500 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.532470    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-dshm\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.532728 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.532709    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f067560b-975b-4922-acaa-4b06d0c69e05-tls-certs\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.538524 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.538498    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5wbq\" (UniqueName: \"kubernetes.io/projected/f067560b-975b-4922-acaa-4b06d0c69e05-kube-api-access-w5wbq\") pod \"rate-test-model-da474e-kserve-7fd6c8fc55-54dvq\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") " pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.608518 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.608453    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:46:51.949365 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:51.949335    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"]
Jul 29 01:46:51.951252 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:46:51.951222    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf067560b_975b_4922_acaa_4b06d0c69e05.slice/crio-ba22f4de6b8f5d7b08911b7997c61abb6ebe7c89c809598648fd702921272421 WatchSource:0}: Error finding container ba22f4de6b8f5d7b08911b7997c61abb6ebe7c89c809598648fd702921272421: Status 404 returned error can't find the container with id ba22f4de6b8f5d7b08911b7997c61abb6ebe7c89c809598648fd702921272421
Jul 29 01:46:52.337192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:52.337091    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq" event={"ID":"f067560b-975b-4922-acaa-4b06d0c69e05","Type":"ContainerStarted","Data":"b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d"}
Jul 29 01:46:52.337192 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:52.337148    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq" event={"ID":"f067560b-975b-4922-acaa-4b06d0c69e05","Type":"ContainerStarted","Data":"ba22f4de6b8f5d7b08911b7997c61abb6ebe7c89c809598648fd702921272421"}
Jul 29 01:46:52.354612 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:46:52.354535    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq" podStartSLOduration=1.3545199 podStartE2EDuration="1.3545199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:46:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:46:52.353607942 +0000 UTC m=+3141.684257548" watchObservedRunningTime="2026-07-29 01:46:52.3545199 +0000 UTC m=+3141.685169505"
Jul 29 01:47:01.609270 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:47:01.609233    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:47:01.609747 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:47:01.609303    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:47:01.622397 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:47:01.622367    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:47:02.401174 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:47:02.401140    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:47:06.924801 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:47:06.924766    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:01.579331 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:01.579251    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:02.377635 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:02.377604    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"]
Jul 29 01:48:02.387694 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:02.387667    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:03.871059 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:03.871028    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:04.096066 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:04.096034    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:16.565421 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:16.565387    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:17.397028 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.396980    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq" podUID="f067560b-975b-4922-acaa-4b06d0c69e05" containerName="main" containerID="cri-o://b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d" gracePeriod=45
Jul 29 01:48:17.650945 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.650872    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:48:17.751673 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751636    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-dshm\") pod \"f067560b-975b-4922-acaa-4b06d0c69e05\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") "
Jul 29 01:48:17.751830 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751689    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-home\") pod \"f067560b-975b-4922-acaa-4b06d0c69e05\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") "
Jul 29 01:48:17.751830 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751733    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-tmp-dir\") pod \"f067560b-975b-4922-acaa-4b06d0c69e05\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") "
Jul 29 01:48:17.751830 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751778    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5wbq\" (UniqueName: \"kubernetes.io/projected/f067560b-975b-4922-acaa-4b06d0c69e05-kube-api-access-w5wbq\") pod \"f067560b-975b-4922-acaa-4b06d0c69e05\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") "
Jul 29 01:48:17.752007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751840    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-model-cache\") pod \"f067560b-975b-4922-acaa-4b06d0c69e05\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") "
Jul 29 01:48:17.752007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751873    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f067560b-975b-4922-acaa-4b06d0c69e05-tls-certs\") pod \"f067560b-975b-4922-acaa-4b06d0c69e05\" (UID: \"f067560b-975b-4922-acaa-4b06d0c69e05\") "
Jul 29 01:48:17.752007 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.751994    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-home" (OuterVolumeSpecName: "home") pod "f067560b-975b-4922-acaa-4b06d0c69e05" (UID: "f067560b-975b-4922-acaa-4b06d0c69e05"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:17.752166 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.752011    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f067560b-975b-4922-acaa-4b06d0c69e05" (UID: "f067560b-975b-4922-acaa-4b06d0c69e05"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:17.752166 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.752130    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-model-cache" (OuterVolumeSpecName: "model-cache") pod "f067560b-975b-4922-acaa-4b06d0c69e05" (UID: "f067560b-975b-4922-acaa-4b06d0c69e05"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:17.752299 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.752264    2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-home\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:48:17.752347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.752301    2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-tmp-dir\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:48:17.752347 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.752312    2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-model-cache\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:48:17.754115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.754090    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f067560b-975b-4922-acaa-4b06d0c69e05-kube-api-access-w5wbq" (OuterVolumeSpecName: "kube-api-access-w5wbq") pod "f067560b-975b-4922-acaa-4b06d0c69e05" (UID: "f067560b-975b-4922-acaa-4b06d0c69e05"). InnerVolumeSpecName "kube-api-access-w5wbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:48:17.754115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.754108    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-dshm" (OuterVolumeSpecName: "dshm") pod "f067560b-975b-4922-acaa-4b06d0c69e05" (UID: "f067560b-975b-4922-acaa-4b06d0c69e05"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:48:17.754361 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.754337    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f067560b-975b-4922-acaa-4b06d0c69e05-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f067560b-975b-4922-acaa-4b06d0c69e05" (UID: "f067560b-975b-4922-acaa-4b06d0c69e05"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:48:17.783699 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.783669    2555 generic.go:356] "Generic (PLEG): container finished" podID="f067560b-975b-4922-acaa-4b06d0c69e05" containerID="b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d" exitCode=0
Jul 29 01:48:17.783863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.783714    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"
Jul 29 01:48:17.783863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.783753    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq" event={"ID":"f067560b-975b-4922-acaa-4b06d0c69e05","Type":"ContainerDied","Data":"b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d"}
Jul 29 01:48:17.783863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.783794    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq" event={"ID":"f067560b-975b-4922-acaa-4b06d0c69e05","Type":"ContainerDied","Data":"ba22f4de6b8f5d7b08911b7997c61abb6ebe7c89c809598648fd702921272421"}
Jul 29 01:48:17.783863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.783811    2555 scope.go:117] "RemoveContainer" containerID="b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d"
Jul 29 01:48:17.793512 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.793493    2555 scope.go:117] "RemoveContainer" containerID="b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d"
Jul 29 01:48:17.793742 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:48:17.793725    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d\": container with ID starting with b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d not found: ID does not exist" containerID="b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d"
Jul 29 01:48:17.793796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.793750    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d"} err="failed to get container status \"b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d\": rpc error: code = NotFound desc = could not find container \"b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d\": container with ID starting with b3764ec58c5804a788a1d9b71862b7689f1b4b3e8f194e94257e81fc3340405d not found: ID does not exist"
Jul 29 01:48:17.806387 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.806361    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"]
Jul 29 01:48:17.808310 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.808291    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-da474e/rate-test-model-da474e-kserve-7fd6c8fc55-54dvq"]
Jul 29 01:48:17.853479 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.853453    2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f067560b-975b-4922-acaa-4b06d0c69e05-dshm\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:48:17.853591 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.853481    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5wbq\" (UniqueName: \"kubernetes.io/projected/f067560b-975b-4922-acaa-4b06d0c69e05-kube-api-access-w5wbq\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:48:17.853591 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:17.853496    2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f067560b-975b-4922-acaa-4b06d0c69e05-tls-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:48:19.190119 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:19.190084    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f067560b-975b-4922-acaa-4b06d0c69e05" path="/var/lib/kubelet/pods/f067560b-975b-4922-acaa-4b06d0c69e05/volumes"
Jul 29 01:48:22.126091 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.126060    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:22.395797 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.395703    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"]
Jul 29 01:48:22.396397 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.396372    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f067560b-975b-4922-acaa-4b06d0c69e05" containerName="main"
Jul 29 01:48:22.396397 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.396394    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f067560b-975b-4922-acaa-4b06d0c69e05" containerName="main"
Jul 29 01:48:22.396572 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.396507    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f067560b-975b-4922-acaa-4b06d0c69e05" containerName="main"
Jul 29 01:48:22.401242 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.401219    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.404305 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.404267    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-295648-openshift-default-dockercfg-7k5lh\""
Jul 29 01:48:22.411796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.411771    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"]
Jul 29 01:48:22.423705 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.423683    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:22.496162 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496117    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-socket\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496162 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496161    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422058f1-2f04-4672-b5a4-e6639c779b95-istio-podinfo\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496187    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-envoy\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496221    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-certs\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496255    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-istio-token\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496272    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-data\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496313    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422058f1-2f04-4672-b5a4-e6639c779b95-istiod-ca-cert\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496334    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-credential-socket\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.496445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.496354    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpv2q\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-kube-api-access-zpv2q\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597612 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597576    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpv2q\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-kube-api-access-zpv2q\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597758 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597740    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-socket\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597775    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422058f1-2f04-4672-b5a4-e6639c779b95-istio-podinfo\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597811    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-envoy\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597930 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597851    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-certs\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597930 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597901    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-istio-token\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.597930 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597925    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-data\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598094 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597958    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422058f1-2f04-4672-b5a4-e6639c779b95-istiod-ca-cert\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598094 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.597993    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-credential-socket\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598264 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.598247    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-socket\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.598416    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-data\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598527 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.598510    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-certs\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598587 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.598512    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-credential-socket\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.598748 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.598720    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422058f1-2f04-4672-b5a4-e6639c779b95-istiod-ca-cert\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.600229 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.600197    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-envoy\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.600554 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.600532    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422058f1-2f04-4672-b5a4-e6639c779b95-istio-podinfo\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.606166 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.606139    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpv2q\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-kube-api-access-zpv2q\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.606259 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.606210    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-istio-token\") pod \"e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") " pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.708048 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.708017    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:22.712724 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.712704    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:22.855640 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.855615    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"]
Jul 29 01:48:22.857126 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:48:22.857091    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422058f1_2f04_4672_b5a4_e6639c779b95.slice/crio-fbc16a4d573b997c449baa8578afd54c852a5f7d4a6a994fa993a4992600651f WatchSource:0}: Error finding container fbc16a4d573b997c449baa8578afd54c852a5f7d4a6a994fa993a4992600651f: Status 404 returned error can't find the container with id fbc16a4d573b997c449baa8578afd54c852a5f7d4a6a994fa993a4992600651f
Jul 29 01:48:22.859376 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.859339    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:22.859470 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.859424    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:22.859470 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:22.859451    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:23.817487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:23.817451    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" event={"ID":"422058f1-2f04-4672-b5a4-e6639c779b95","Type":"ContainerStarted","Data":"47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"}
Jul 29 01:48:23.817929 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:23.817509    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" event={"ID":"422058f1-2f04-4672-b5a4-e6639c779b95","Type":"ContainerStarted","Data":"fbc16a4d573b997c449baa8578afd54c852a5f7d4a6a994fa993a4992600651f"}
Jul 29 01:48:23.841729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:23.841673    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" podStartSLOduration=1.841654793 podStartE2EDuration="1.841654793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:48:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:48:23.838675937 +0000 UTC m=+3233.169325543" watchObservedRunningTime="2026-07-29 01:48:23.841654793 +0000 UTC m=+3233.172304398"
Jul 29 01:48:24.712893 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:24.712850    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:24.717932 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:24.717907    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:24.822872 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:24.822834    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:24.823733 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:24.823713    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:25.738382 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:25.738335    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:26.956653 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:26.956613    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:27.056826 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:27.056795    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:40.864467 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:40.864426    2555 patch_prober.go:28] interesting pod/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 29 01:48:40.864873 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:40.864507    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 29 01:48:42.922306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:42.922256    2555 generic.go:356] "Generic (PLEG): container finished" podID="422058f1-2f04-4672-b5a4-e6639c779b95" containerID="47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f" exitCode=0
Jul 29 01:48:42.922718 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:42.922332    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" event={"ID":"422058f1-2f04-4672-b5a4-e6639c779b95","Type":"ContainerDied","Data":"47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"}
Jul 29 01:48:42.922761 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:42.922737    2555 scope.go:117] "RemoveContainer" containerID="47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"
Jul 29 01:48:42.923796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:42.923768    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:42.923873 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:42.923843    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:42.923953 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:42.923936    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:43.929809 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:43.929767    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" event={"ID":"422058f1-2f04-4672-b5a4-e6639c779b95","Type":"ContainerStarted","Data":"1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542"}
Jul 29 01:48:44.713731 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:44.713690    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:44.713982 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:44.713955    2555 patch_prober.go:28] interesting pod/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Jul 29 01:48:44.714073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:44.714015    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Jul 29 01:48:45.713516 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:45.713488    2555 patch_prober.go:28] interesting pod/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Jul 29 01:48:45.713883 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:45.713542    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Jul 29 01:48:46.713752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:46.713711    2555 patch_prober.go:28] interesting pod/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused" start-of-body=
Jul 29 01:48:46.714181 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:46.713789    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.88:15021/healthz/ready\": dial tcp 10.134.0.88:15021: connect: connection refused"
Jul 29 01:48:47.716679 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:47.716646    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:47.717235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:47.717032    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:47.717762 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:47.717742    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:48:50.713440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:50.713408    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:51.374188 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.374158    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:51.703796 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.703763    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"]
Jul 29 01:48:51.707243 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.707227    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.710660 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.710639    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-8230fd-openshift-default-dockercfg-8fdxj\""
Jul 29 01:48:51.721629 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.721584    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"]
Jul 29 01:48:51.733566 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.733540    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:51.791606 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791575    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-credential-socket\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.791803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791619    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64394b9d-5a6b-4d60-9522-4b465b15549e-istiod-ca-cert\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.791803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791645    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-data\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.791803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791716    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-podinfo\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.791960 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791814    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xf9\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-kube-api-access-v4xf9\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.791960 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791925    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-socket\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.792044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791977    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-certs\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.792044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.791994    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-token\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.792044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.792021    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-envoy\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892671 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892631    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-socket\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892681    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-certs\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892699    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-token\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892738    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-envoy\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892782    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-credential-socket\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892815    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64394b9d-5a6b-4d60-9522-4b465b15549e-istiod-ca-cert\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.892860 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892843    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-data\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892868    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-podinfo\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.892899    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xf9\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-kube-api-access-v4xf9\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893126 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.893105    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-socket\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893328 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.893160    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-certs\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893393 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.893324    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-credential-socket\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.893385    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-data\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.893593 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.893565    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64394b9d-5a6b-4d60-9522-4b465b15549e-istiod-ca-cert\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.895373 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.895346    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-envoy\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.895529 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.895512    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-podinfo\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.903476 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.903454    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-token\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:51.903931 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:51.903913    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xf9\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-kube-api-access-v4xf9\") pod \"e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") " pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:52.018692 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.018596    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:52.164979 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.164950    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"]
Jul 29 01:48:52.166224 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:48:52.166196    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64394b9d_5a6b_4d60_9522_4b465b15549e.slice/crio-2af32b8e15c1496459a7f37b9bf8a0e2e15f3f353a10d5dd7360a86e83aa4ed6 WatchSource:0}: Error finding container 2af32b8e15c1496459a7f37b9bf8a0e2e15f3f353a10d5dd7360a86e83aa4ed6: Status 404 returned error can't find the container with id 2af32b8e15c1496459a7f37b9bf8a0e2e15f3f353a10d5dd7360a86e83aa4ed6
Jul 29 01:48:52.168487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.168456    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:52.168551 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.168540    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:52.168590 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.168577    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:48:52.170368 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.170348    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:52.995911 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.995873    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" event={"ID":"64394b9d-5a6b-4d60-9522-4b465b15549e","Type":"ContainerStarted","Data":"26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"}
Jul 29 01:48:52.996310 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:52.995936    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" event={"ID":"64394b9d-5a6b-4d60-9522-4b465b15549e","Type":"ContainerStarted","Data":"2af32b8e15c1496459a7f37b9bf8a0e2e15f3f353a10d5dd7360a86e83aa4ed6"}
Jul 29 01:48:53.018734 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:53.018702    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:53.018866 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:53.018770    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" podStartSLOduration=2.018754518 podStartE2EDuration="2.018754518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:48:53.015794046 +0000 UTC m=+3262.346443654" watchObservedRunningTime="2026-07-29 01:48:53.018754518 +0000 UTC m=+3262.349404128"
Jul 29 01:48:53.023957 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:53.023936    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:54.001776 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:54.001742    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:54.002798 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:54.002779    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:48:54.947768 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:54.947733    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:55.047044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.047013    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:55.128542 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.128506    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:55.670921 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.670865    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:55.726567 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.726533    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"]
Jul 29 01:48:55.730291 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.730259    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:55.733124 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.733094    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-8230fd-dockercfg-xrzzq\""
Jul 29 01:48:55.733265 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.733121    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-8230fd\""
Jul 29 01:48:55.739152 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.738770    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"]
Jul 29 01:48:55.830715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.830469    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/717b7270-61a1-4a98-b86d-aaf51c38fd8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:55.830715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.830525    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7scc\" (UniqueName: \"kubernetes.io/projected/717b7270-61a1-4a98-b86d-aaf51c38fd8e-kube-api-access-z7scc\") pod \"payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:55.931951 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.931863    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/717b7270-61a1-4a98-b86d-aaf51c38fd8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:55.931951 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.931903    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7scc\" (UniqueName: \"kubernetes.io/projected/717b7270-61a1-4a98-b86d-aaf51c38fd8e-kube-api-access-z7scc\") pod \"payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:55.932807 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.932781    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/717b7270-61a1-4a98-b86d-aaf51c38fd8e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:55.942697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:55.942672    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7scc\" (UniqueName: \"kubernetes.io/projected/717b7270-61a1-4a98-b86d-aaf51c38fd8e-kube-api-access-z7scc\") pod \"payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:56.042640 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:56.042600    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:56.190076 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:56.190045    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"]
Jul 29 01:48:56.191580 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:48:56.191554    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717b7270_61a1_4a98_b86d_aaf51c38fd8e.slice/crio-0de05147e356fdc7e105cfb2d97a90517ff96e9f8e5eedf30d9e75971e105fe1 WatchSource:0}: Error finding container 0de05147e356fdc7e105cfb2d97a90517ff96e9f8e5eedf30d9e75971e105fe1: Status 404 returned error can't find the container with id 0de05147e356fdc7e105cfb2d97a90517ff96e9f8e5eedf30d9e75971e105fe1
Jul 29 01:48:56.820434 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:56.820399    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:56.921207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:56.921173    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:48:57.018919 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:57.018881    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt" event={"ID":"717b7270-61a1-4a98-b86d-aaf51c38fd8e","Type":"ContainerStarted","Data":"22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8"}
Jul 29 01:48:57.019115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:57.018944    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:48:57.019115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:57.018968    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt" event={"ID":"717b7270-61a1-4a98-b86d-aaf51c38fd8e","Type":"ContainerStarted","Data":"0de05147e356fdc7e105cfb2d97a90517ff96e9f8e5eedf30d9e75971e105fe1"}
Jul 29 01:48:57.040046 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:48:57.039981    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt" podStartSLOduration=2.039966316 podStartE2EDuration="2.039966316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:48:57.037849321 +0000 UTC m=+3266.368498928" watchObservedRunningTime="2026-07-29 01:48:57.039966316 +0000 UTC m=+3266.370615921"
Jul 29 01:49:08.026440 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:08.026406    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:49:13.124387 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:13.124353    2555 generic.go:356] "Generic (PLEG): container finished" podID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerID="26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc" exitCode=0
Jul 29 01:49:13.124808 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:13.124425    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" event={"ID":"64394b9d-5a6b-4d60-9522-4b465b15549e","Type":"ContainerDied","Data":"26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"}
Jul 29 01:49:13.124808 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:13.124800    2555 scope.go:117] "RemoveContainer" containerID="26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"
Jul 29 01:49:13.125863 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:13.125835    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:49:13.125910 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:13.125902    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:49:13.125945 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:13.125932    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:49:14.132346 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:14.132303    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" event={"ID":"64394b9d-5a6b-4d60-9522-4b465b15549e","Type":"ContainerStarted","Data":"1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22"}
Jul 29 01:49:15.018725 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:15.018689    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:49:15.019029 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:15.019003    2555 patch_prober.go:28] interesting pod/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Jul 29 01:49:15.019086 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:15.019059    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Jul 29 01:49:16.019441 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:16.019403    2555 patch_prober.go:28] interesting pod/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Jul 29 01:49:16.019849 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:16.019458    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Jul 29 01:49:17.019857 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:17.019820    2555 patch_prober.go:28] interesting pod/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused" start-of-body=
Jul 29 01:49:17.020296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:17.019870    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.89:15021/healthz/ready\": dial tcp 10.134.0.89:15021: connect: connection refused"
Jul 29 01:49:18.022266 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:18.022233    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:49:18.022641 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:18.022601    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:49:18.023304 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:18.023258    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:49:19.969019 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:19.968985    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:49:26.710994 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:26.710965    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-z8k9t_83855546-55fb-4f87-948b-086b183adc64/payload-processing/0.log"
Jul 29 01:49:33.095835 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:49:33.095791    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:13.726493 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:13.726403    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-z8k9t_83855546-55fb-4f87-948b-086b183adc64/payload-processing/0.log"
Jul 29 01:50:14.113794 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.113715    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:14.389359 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.389264    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"]
Jul 29 01:50:14.393639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.393617    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.396325 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.396294    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-dc8b9a-openshift-default-dockercfg-p9g52\""
Jul 29 01:50:14.405267 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.405242    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"]
Jul 29 01:50:14.505602 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505571    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-data\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505606    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f721afc5-0487-4360-9d01-7dcf8378211d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505640    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-istio-token\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505695    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f721afc5-0487-4360-9d01-7dcf8378211d-istio-podinfo\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505737    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2m75\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-kube-api-access-g2m75\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505779    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-credential-socket\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505816    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-certs\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.505909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505840    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-socket\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.506004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.505937    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-envoy\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.606770 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.606736    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-socket\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.606940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.606810    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-envoy\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.606940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.606862    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-data\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.606940 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.606885    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f721afc5-0487-4360-9d01-7dcf8378211d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607107 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607018    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-istio-token\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607107 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607065    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f721afc5-0487-4360-9d01-7dcf8378211d-istio-podinfo\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607107 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607093    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2m75\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-kube-api-access-g2m75\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607136    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-credential-socket\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607189    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-certs\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607238    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-socket\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607515 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607495    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-credential-socket\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607612 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607589    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-certs\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607674 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607628    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f721afc5-0487-4360-9d01-7dcf8378211d-istiod-ca-cert\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.607723 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.607709    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-data\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.609111 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.609085    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-envoy\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.609478 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.609461    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f721afc5-0487-4360-9d01-7dcf8378211d-istio-podinfo\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.614800 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.614779    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-istio-token\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.614943 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.614927    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2m75\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-kube-api-access-g2m75\") pod \"e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") " pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.706191 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.706160    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:14.872072 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.872040    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"]
Jul 29 01:50:14.874002 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:50:14.873975    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf721afc5_0487_4360_9d01_7dcf8378211d.slice/crio-c6aaaf2f93cf5f57a30ad9d25b1927732d1c86848c81ceb959ea04dfd3481ec8 WatchSource:0}: Error finding container c6aaaf2f93cf5f57a30ad9d25b1927732d1c86848c81ceb959ea04dfd3481ec8: Status 404 returned error can't find the container with id c6aaaf2f93cf5f57a30ad9d25b1927732d1c86848c81ceb959ea04dfd3481ec8
Jul 29 01:50:14.876207 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.876177    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:50:14.876269 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.876240    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:50:14.876269 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:14.876265    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:50:15.475132 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:15.475096    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" event={"ID":"f721afc5-0487-4360-9d01-7dcf8378211d","Type":"ContainerStarted","Data":"635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"}
Jul 29 01:50:15.475340 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:15.475158    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" event={"ID":"f721afc5-0487-4360-9d01-7dcf8378211d","Type":"ContainerStarted","Data":"c6aaaf2f93cf5f57a30ad9d25b1927732d1c86848c81ceb959ea04dfd3481ec8"}
Jul 29 01:50:15.499211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:15.499158    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podStartSLOduration=1.499142193 podStartE2EDuration="1.499142193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:50:15.496585733 +0000 UTC m=+3344.827235330" watchObservedRunningTime="2026-07-29 01:50:15.499142193 +0000 UTC m=+3344.829791797"
Jul 29 01:50:15.706887 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:15.706855    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:15.712169 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:15.712143    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:16.448363 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:16.448326    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:16.480445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:16.480424    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:16.481332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:16.481313    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:17.466608 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:17.466577    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:17.556826 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:17.556788    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:17.745590 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:17.745501    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:18.350909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:18.350882    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:18.450584 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:18.450547    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:35.597251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:35.597214    2555 generic.go:356] "Generic (PLEG): container finished" podID="f721afc5-0487-4360-9d01-7dcf8378211d" containerID="635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8" exitCode=0
Jul 29 01:50:35.597704 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:35.597310    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" event={"ID":"f721afc5-0487-4360-9d01-7dcf8378211d","Type":"ContainerDied","Data":"635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"}
Jul 29 01:50:35.597752 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:35.597721    2555 scope.go:117] "RemoveContainer" containerID="635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"
Jul 29 01:50:35.598928 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:35.598903    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:50:35.599024 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:35.599007    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:50:35.599064 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:35.599054    2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 01:50:36.604050 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:36.604010    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" event={"ID":"f721afc5-0487-4360-9d01-7dcf8378211d","Type":"ContainerStarted","Data":"b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f"}
Jul 29 01:50:36.706669 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:36.706629    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:36.706960 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:36.706932    2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 29 01:50:36.707067 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:36.706990    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 29 01:50:37.707141 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:37.707100    2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 29 01:50:37.707630 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:37.707159    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 29 01:50:38.706782 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:38.706743    2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 29 01:50:38.707041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:38.706804    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 29 01:50:39.706907 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:39.706868    2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Jul 29 01:50:39.707331 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:39.706924    2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Jul 29 01:50:40.710506 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:40.710475    2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:40.710992 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:40.710871    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:40.711507 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:40.711489    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:50:42.288636 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:42.288603    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:48.240200 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:48.240157    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:48.312861 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:48.312820    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:48.411842 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:48.411807    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:48.479862 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:48.479834    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:48.776196 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:48.776168    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:54.741746 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:54.741716    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:54.845246 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:54.845207    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:54.972916 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:54.972880    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"]
Jul 29 01:50:54.973198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:54.973172    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy" containerID="cri-o://b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f" gracePeriod=30
Jul 29 01:50:55.000264 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:55.000196    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:55.276487 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:55.276410    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:50:55.712869 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:55.712840    2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 01:50:55.713027 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:50:55.712902    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 01:51:00.242476 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.242402    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:51:00.372409 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372375    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-envoy\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372421    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-data\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372441    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-certs\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372465    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-credential-socket\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372502    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2m75\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-kube-api-access-g2m75\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372577 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372520    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-istio-token\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372572    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-socket\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372662    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f721afc5-0487-4360-9d01-7dcf8378211d-istiod-ca-cert\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372722    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f721afc5-0487-4360-9d01-7dcf8378211d-istio-podinfo\") pod \"f721afc5-0487-4360-9d01-7dcf8378211d\" (UID: \"f721afc5-0487-4360-9d01-7dcf8378211d\") "
Jul 29 01:51:00.372824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372742    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:00.373030 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.372824    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:00.373100 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.373048    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f721afc5-0487-4360-9d01-7dcf8378211d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:51:00.373164 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.373101    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:00.373220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.373164    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.373220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.373180    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-credential-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.373220 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.373193    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f721afc5-0487-4360-9d01-7dcf8378211d-istiod-ca-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.373405 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.373387    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:00.374757 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.374734    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f721afc5-0487-4360-9d01-7dcf8378211d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:51:00.374870 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.374798    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:51:00.374951 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.374935    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:51:00.375032 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.375014    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-kube-api-access-g2m75" (OuterVolumeSpecName: "kube-api-access-g2m75") pod "f721afc5-0487-4360-9d01-7dcf8378211d" (UID: "f721afc5-0487-4360-9d01-7dcf8378211d"). InnerVolumeSpecName "kube-api-access-g2m75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:51:00.474462 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.474433    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2m75\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-kube-api-access-g2m75\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.474462 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.474457    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f721afc5-0487-4360-9d01-7dcf8378211d-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.474462 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.474466    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-workload-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.474650 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.474474    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f721afc5-0487-4360-9d01-7dcf8378211d-istio-podinfo\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.474650 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.474482    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-envoy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.474650 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.474489    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f721afc5-0487-4360-9d01-7dcf8378211d-istio-data\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:51:00.736201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.736164    2555 generic.go:356] "Generic (PLEG): container finished" podID="f721afc5-0487-4360-9d01-7dcf8378211d" containerID="b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f" exitCode=0
Jul 29 01:51:00.736438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.736218    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"
Jul 29 01:51:00.736438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.736261    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" event={"ID":"f721afc5-0487-4360-9d01-7dcf8378211d","Type":"ContainerDied","Data":"b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f"}
Jul 29 01:51:00.736438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.736390    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4" event={"ID":"f721afc5-0487-4360-9d01-7dcf8378211d","Type":"ContainerDied","Data":"c6aaaf2f93cf5f57a30ad9d25b1927732d1c86848c81ceb959ea04dfd3481ec8"}
Jul 29 01:51:00.736438 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.736423    2555 scope.go:117] "RemoveContainer" containerID="b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f"
Jul 29 01:51:00.746648 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.746629    2555 scope.go:117] "RemoveContainer" containerID="635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"
Jul 29 01:51:00.754660 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.754642    2555 scope.go:117] "RemoveContainer" containerID="b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f"
Jul 29 01:51:00.754911 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:51:00.754892    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f\": container with ID starting with b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f not found: ID does not exist" containerID="b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f"
Jul 29 01:51:00.754958 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.754919    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f"} err="failed to get container status \"b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f\": rpc error: code = NotFound desc = could not find container \"b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f\": container with ID starting with b0e6e9c12bb453daa1f6c597a478d229e77e18bc09ca646b93c6970bd1abde5f not found: ID does not exist"
Jul 29 01:51:00.754958 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.754935    2555 scope.go:117] "RemoveContainer" containerID="635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"
Jul 29 01:51:00.755159 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:51:00.755144    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8\": container with ID starting with 635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8 not found: ID does not exist" containerID="635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"
Jul 29 01:51:00.755201 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.755165    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8"} err="failed to get container status \"635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8\": rpc error: code = NotFound desc = could not find container \"635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8\": container with ID starting with 635b7f69c179aaad0106deda9b9023bde4b3a9e130d2686badf4b02948f934e8 not found: ID does not exist"
Jul 29 01:51:00.759824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.759804    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"]
Jul 29 01:51:00.763818 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:00.763798    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-dc8b9a-openshift-default-c8ff56498-xlmz4"]
Jul 29 01:51:01.190813 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:01.190774    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" path="/var/lib/kubelet/pods/f721afc5-0487-4360-9d01-7dcf8378211d/volumes"
Jul 29 01:51:01.879227 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:01.879197    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:14.071495 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:14.071450    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:14.174991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:14.174959    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:14.274527 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:14.274489    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:14.349643 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:14.349559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:14.675797 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:14.675719    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:19.732611 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:19.732572    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:55.914651 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:55.914621    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:51:57.180713 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:51:57.180676    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:02.956707 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:02.956648    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:03.058093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:03.058055    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:03.211297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:03.211197    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"]
Jul 29 01:52:03.211589 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:03.211557    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy" containerID="cri-o://1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542" gracePeriod=30
Jul 29 01:52:03.234800 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:03.234766    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:03.454325 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:03.454259    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:08.482027 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.481960    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:52:08.532722 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532686    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-certs\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532722 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532725    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422058f1-2f04-4672-b5a4-e6639c779b95-istiod-ca-cert\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532748    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422058f1-2f04-4672-b5a4-e6639c779b95-istio-podinfo\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532773    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-istio-token\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532830    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-envoy\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532858    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-credential-socket\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532898    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-data\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532926    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpv2q\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-kube-api-access-zpv2q\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.532986 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.532950    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-socket\") pod \"422058f1-2f04-4672-b5a4-e6639c779b95\" (UID: \"422058f1-2f04-4672-b5a4-e6639c779b95\") "
Jul 29 01:52:08.533403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533062    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:08.533403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533171    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/422058f1-2f04-4672-b5a4-e6639c779b95-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:08.533403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533322    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.533403 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533341    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/422058f1-2f04-4672-b5a4-e6639c779b95-istiod-ca-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.533624 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533528    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:08.533731 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533713    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:08.533945 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.533901    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-data" (OuterVolumeSpecName: "istio-data") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:08.535297 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.535251    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:08.535598 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.535563    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-istio-token" (OuterVolumeSpecName: "istio-token") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:08.535824 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.535804    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/422058f1-2f04-4672-b5a4-e6639c779b95-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:52:08.535896 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.535823    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-kube-api-access-zpv2q" (OuterVolumeSpecName: "kube-api-access-zpv2q") pod "422058f1-2f04-4672-b5a4-e6639c779b95" (UID: "422058f1-2f04-4672-b5a4-e6639c779b95"). InnerVolumeSpecName "kube-api-access-zpv2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:08.634520 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634481    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-data\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.634520 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634513    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpv2q\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-kube-api-access-zpv2q\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.634520 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634524    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-workload-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.634520 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634533    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/422058f1-2f04-4672-b5a4-e6639c779b95-istio-podinfo\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.634888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634541    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/422058f1-2f04-4672-b5a4-e6639c779b95-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.634888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634549    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-istio-envoy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:08.634888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:08.634556    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/422058f1-2f04-4672-b5a4-e6639c779b95-credential-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:09.119153 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.119118    2555 generic.go:356] "Generic (PLEG): container finished" podID="422058f1-2f04-4672-b5a4-e6639c779b95" containerID="1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542" exitCode=0
Jul 29 01:52:09.119370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.119168    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"
Jul 29 01:52:09.119370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.119205    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" event={"ID":"422058f1-2f04-4672-b5a4-e6639c779b95","Type":"ContainerDied","Data":"1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542"}
Jul 29 01:52:09.119370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.119263    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s" event={"ID":"422058f1-2f04-4672-b5a4-e6639c779b95","Type":"ContainerDied","Data":"fbc16a4d573b997c449baa8578afd54c852a5f7d4a6a994fa993a4992600651f"}
Jul 29 01:52:09.119370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.119310    2555 scope.go:117] "RemoveContainer" containerID="1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542"
Jul 29 01:52:09.129826 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.129808    2555 scope.go:117] "RemoveContainer" containerID="47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"
Jul 29 01:52:09.139620 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.139589    2555 scope.go:117] "RemoveContainer" containerID="1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542"
Jul 29 01:52:09.139931 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:09.139910    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542\": container with ID starting with 1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542 not found: ID does not exist" containerID="1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542"
Jul 29 01:52:09.140013 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.139938    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542"} err="failed to get container status \"1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542\": rpc error: code = NotFound desc = could not find container \"1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542\": container with ID starting with 1781c974a41319b17437cb0e6d2c36cc01886f76c9bae28393681c8798f83542 not found: ID does not exist"
Jul 29 01:52:09.140013 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.139958    2555 scope.go:117] "RemoveContainer" containerID="47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"
Jul 29 01:52:09.140187 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:09.140166    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f\": container with ID starting with 47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f not found: ID does not exist" containerID="47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"
Jul 29 01:52:09.140235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.140194    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f"} err="failed to get container status \"47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f\": rpc error: code = NotFound desc = could not find container \"47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f\": container with ID starting with 47ab88cce439b02ba81fe66b4287c1b25519f9ea7b5d345099518c3b00ac877f not found: ID does not exist"
Jul 29 01:52:09.144639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.144614    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"]
Jul 29 01:52:09.149441 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.149414    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-295648-openshift-default-767bf98b6b-r4m5s"]
Jul 29 01:52:09.190627 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:09.190596    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" path="/var/lib/kubelet/pods/422058f1-2f04-4672-b5a4-e6639c779b95/volumes"
Jul 29 01:52:14.188057 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.188020    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:14.221341 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.221306    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"]
Jul 29 01:52:14.221623 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.221593    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt" podUID="717b7270-61a1-4a98-b86d-aaf51c38fd8e" containerName="payload-pre-processing" containerID="cri-o://22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8" gracePeriod=30
Jul 29 01:52:14.288905 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.288868    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:14.368310 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.368256    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:14.430528 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.430474    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:14.492396 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.492366    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:52:14.594123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.594088    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7scc\" (UniqueName: \"kubernetes.io/projected/717b7270-61a1-4a98-b86d-aaf51c38fd8e-kube-api-access-z7scc\") pod \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") "
Jul 29 01:52:14.594123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.594120    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/717b7270-61a1-4a98-b86d-aaf51c38fd8e-plugins-config-volume\") pod \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\" (UID: \"717b7270-61a1-4a98-b86d-aaf51c38fd8e\") "
Jul 29 01:52:14.594598 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.594576    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/717b7270-61a1-4a98-b86d-aaf51c38fd8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "717b7270-61a1-4a98-b86d-aaf51c38fd8e" (UID: "717b7270-61a1-4a98-b86d-aaf51c38fd8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:14.596363 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.596338    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/717b7270-61a1-4a98-b86d-aaf51c38fd8e-kube-api-access-z7scc" (OuterVolumeSpecName: "kube-api-access-z7scc") pod "717b7270-61a1-4a98-b86d-aaf51c38fd8e" (UID: "717b7270-61a1-4a98-b86d-aaf51c38fd8e"). InnerVolumeSpecName "kube-api-access-z7scc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:14.695959 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.695925    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7scc\" (UniqueName: \"kubernetes.io/projected/717b7270-61a1-4a98-b86d-aaf51c38fd8e-kube-api-access-z7scc\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:14.695959 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.695954    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/717b7270-61a1-4a98-b86d-aaf51c38fd8e-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:14.812648 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:14.812568    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:15.155749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.155628    2555 generic.go:356] "Generic (PLEG): container finished" podID="717b7270-61a1-4a98-b86d-aaf51c38fd8e" containerID="22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8" exitCode=0
Jul 29 01:52:15.155749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.155676    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"
Jul 29 01:52:15.155749 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.155717    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt" event={"ID":"717b7270-61a1-4a98-b86d-aaf51c38fd8e","Type":"ContainerDied","Data":"22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8"}
Jul 29 01:52:15.156014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.155789    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt" event={"ID":"717b7270-61a1-4a98-b86d-aaf51c38fd8e","Type":"ContainerDied","Data":"0de05147e356fdc7e105cfb2d97a90517ff96e9f8e5eedf30d9e75971e105fe1"}
Jul 29 01:52:15.156014 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.155829    2555 scope.go:117] "RemoveContainer" containerID="22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8"
Jul 29 01:52:15.164970 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.164807    2555 scope.go:117] "RemoveContainer" containerID="22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8"
Jul 29 01:52:15.165153 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:15.165127    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8\": container with ID starting with 22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8 not found: ID does not exist" containerID="22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8"
Jul 29 01:52:15.165235 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.165164    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8"} err="failed to get container status \"22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8\": rpc error: code = NotFound desc = could not find container \"22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8\": container with ID starting with 22136ce5de58132bc3ba2e7bcf8ccd4c1909795d2fc30c3ec3e0d630f03b44f8 not found: ID does not exist"
Jul 29 01:52:15.179090 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.179068    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"]
Jul 29 01:52:15.182803 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.182781    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8230fd-f7c86794d-x7bnt"]
Jul 29 01:52:15.190047 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:15.190023    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="717b7270-61a1-4a98-b86d-aaf51c38fd8e" path="/var/lib/kubelet/pods/717b7270-61a1-4a98-b86d-aaf51c38fd8e/volumes"
Jul 29 01:52:20.550805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:20.550766    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:20.651553 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:20.651500    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:20.784832 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:20.784800    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"]
Jul 29 01:52:20.785104 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:20.785059    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy" containerID="cri-o://1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22" gracePeriod=30
Jul 29 01:52:20.807642 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:20.807566    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:21.019533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:21.019501    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:26.045004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.044978    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:52:26.110607 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110529    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-podinfo\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110607 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110567    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-credential-socket\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110688    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-data\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110731    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-token\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110805 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110794    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-socket\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110821    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-certs\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110851    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4xf9\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-kube-api-access-v4xf9\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110887    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64394b9d-5a6b-4d60-9522-4b465b15549e-istiod-ca-cert\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.110974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110899    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:26.110974 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.110926    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-envoy\") pod \"64394b9d-5a6b-4d60-9522-4b465b15549e\" (UID: \"64394b9d-5a6b-4d60-9522-4b465b15549e\") "
Jul 29 01:52:26.111236 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111047    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:26.111236 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111112    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:26.111400 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111378    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-credential-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.111480 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111405    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.111480 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111420    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-workload-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.111480 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111449    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64394b9d-5a6b-4d60-9522-4b465b15549e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:26.111626 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.111517    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-data" (OuterVolumeSpecName: "istio-data") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:26.113178 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.113148    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-token" (OuterVolumeSpecName: "istio-token") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:26.113353 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.113204    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:52:26.113533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.113511    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-kube-api-access-v4xf9" (OuterVolumeSpecName: "kube-api-access-v4xf9") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "kube-api-access-v4xf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:26.113634 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.113588    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "64394b9d-5a6b-4d60-9522-4b465b15549e" (UID: "64394b9d-5a6b-4d60-9522-4b465b15549e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:52:26.212252 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.212222    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-podinfo\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.212252 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.212247    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-data\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.212252 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.212255    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.212509 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.212263    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4xf9\" (UniqueName: \"kubernetes.io/projected/64394b9d-5a6b-4d60-9522-4b465b15549e-kube-api-access-v4xf9\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.212509 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.212295    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64394b9d-5a6b-4d60-9522-4b465b15549e-istiod-ca-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.212509 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.212304    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64394b9d-5a6b-4d60-9522-4b465b15549e-istio-envoy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:26.216903 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.216876    2555 generic.go:356] "Generic (PLEG): container finished" podID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerID="1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22" exitCode=0
Jul 29 01:52:26.217018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.216925    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"
Jul 29 01:52:26.217018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.216943    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" event={"ID":"64394b9d-5a6b-4d60-9522-4b465b15549e","Type":"ContainerDied","Data":"1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22"}
Jul 29 01:52:26.217018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.216984    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn" event={"ID":"64394b9d-5a6b-4d60-9522-4b465b15549e","Type":"ContainerDied","Data":"2af32b8e15c1496459a7f37b9bf8a0e2e15f3f353a10d5dd7360a86e83aa4ed6"}
Jul 29 01:52:26.217018 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.217005    2555 scope.go:117] "RemoveContainer" containerID="1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22"
Jul 29 01:52:26.226518 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.226499    2555 scope.go:117] "RemoveContainer" containerID="26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"
Jul 29 01:52:26.236476 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.236458    2555 scope.go:117] "RemoveContainer" containerID="1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22"
Jul 29 01:52:26.236749 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:26.236727    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22\": container with ID starting with 1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22 not found: ID does not exist" containerID="1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22"
Jul 29 01:52:26.236820 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.236759    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22"} err="failed to get container status \"1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22\": rpc error: code = NotFound desc = could not find container \"1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22\": container with ID starting with 1e48a7709efea63efd5092db71c007f2808d498d9b3e0ba4ec7899ad34cf7c22 not found: ID does not exist"
Jul 29 01:52:26.236868 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.236825    2555 scope.go:117] "RemoveContainer" containerID="26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"
Jul 29 01:52:26.237161 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:26.237132    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc\": container with ID starting with 26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc not found: ID does not exist" containerID="26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"
Jul 29 01:52:26.237251 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.237170    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc"} err="failed to get container status \"26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc\": rpc error: code = NotFound desc = could not find container \"26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc\": container with ID starting with 26e7730b818e32106656ba63770d74c835f0cd29a1458848d3ae0625ac9e00cc not found: ID does not exist"
Jul 29 01:52:26.240452 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.240428    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"]
Jul 29 01:52:26.242268 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:26.242250    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-8230fd-openshift-default-77975d4c66-kwrzn"]
Jul 29 01:52:27.190539 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:27.190505    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" path="/var/lib/kubelet/pods/64394b9d-5a6b-4d60-9522-4b465b15549e/volumes"
Jul 29 01:52:31.774587 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:31.774559    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:31.802602 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:31.802564    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"]
Jul 29 01:52:31.802901 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:31.802873    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff" podUID="35ddf2a8-f8f6-4767-8903-2aa696b091db" containerName="payload-processing" containerID="cri-o://fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161" gracePeriod=30
Jul 29 01:52:31.849568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:31.849513    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:31.896525 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:31.896488    2555 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ddf2a8_f8f6_4767_8903_2aa696b091db.slice/crio-conmon-fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161.scope\": RecentStats: unable to find data in memory cache]"
Jul 29 01:52:31.948760 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:31.948721    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:32.010130 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.010094    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:32.070384 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.070358    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:52:32.164652 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.164618    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ddf2a8-f8f6-4767-8903-2aa696b091db-plugins-config-volume\") pod \"35ddf2a8-f8f6-4767-8903-2aa696b091db\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") "
Jul 29 01:52:32.164821 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.164695    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z72wm\" (UniqueName: \"kubernetes.io/projected/35ddf2a8-f8f6-4767-8903-2aa696b091db-kube-api-access-z72wm\") pod \"35ddf2a8-f8f6-4767-8903-2aa696b091db\" (UID: \"35ddf2a8-f8f6-4767-8903-2aa696b091db\") "
Jul 29 01:52:32.165004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.164981    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35ddf2a8-f8f6-4767-8903-2aa696b091db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "35ddf2a8-f8f6-4767-8903-2aa696b091db" (UID: "35ddf2a8-f8f6-4767-8903-2aa696b091db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:32.166670 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.166648    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35ddf2a8-f8f6-4767-8903-2aa696b091db-kube-api-access-z72wm" (OuterVolumeSpecName: "kube-api-access-z72wm") pod "35ddf2a8-f8f6-4767-8903-2aa696b091db" (UID: "35ddf2a8-f8f6-4767-8903-2aa696b091db"). InnerVolumeSpecName "kube-api-access-z72wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:32.249697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.249660    2555 generic.go:356] "Generic (PLEG): container finished" podID="35ddf2a8-f8f6-4767-8903-2aa696b091db" containerID="fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161" exitCode=0
Jul 29 01:52:32.249697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.249698    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"
Jul 29 01:52:32.250198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.249737    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff" event={"ID":"35ddf2a8-f8f6-4767-8903-2aa696b091db","Type":"ContainerDied","Data":"fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161"}
Jul 29 01:52:32.250198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.249793    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff" event={"ID":"35ddf2a8-f8f6-4767-8903-2aa696b091db","Type":"ContainerDied","Data":"741165a5293a4d5be8c7faa9bad576603033a21c28e47ede80bd59b92d66ce98"}
Jul 29 01:52:32.250198 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.249816    2555 scope.go:117] "RemoveContainer" containerID="fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161"
Jul 29 01:52:32.259039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.259019    2555 scope.go:117] "RemoveContainer" containerID="fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161"
Jul 29 01:52:32.259339 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:32.259319    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161\": container with ID starting with fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161 not found: ID does not exist" containerID="fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161"
Jul 29 01:52:32.259411 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.259352    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161"} err="failed to get container status \"fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161\": rpc error: code = NotFound desc = could not find container \"fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161\": container with ID starting with fb34928fd459b570f2fed31bdfa0eba15f227600c71dafecec866bc69e6e7161 not found: ID does not exist"
Jul 29 01:52:32.265951 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.265923    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z72wm\" (UniqueName: \"kubernetes.io/projected/35ddf2a8-f8f6-4767-8903-2aa696b091db-kube-api-access-z72wm\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:32.266058 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.265957    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/35ddf2a8-f8f6-4767-8903-2aa696b091db-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:32.288199 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.288165    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"]
Jul 29 01:52:32.291696 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.291668    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-5ea1e0-55fb6fd7f8-jxzff"]
Jul 29 01:52:32.368525 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:32.368440    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:33.190250 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:33.190215    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35ddf2a8-f8f6-4767-8903-2aa696b091db" path="/var/lib/kubelet/pods/35ddf2a8-f8f6-4767-8903-2aa696b091db/volumes"
Jul 29 01:52:38.251556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:38.251522    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:38.347305 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:38.347246    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:38.511654 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:38.511547    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:38.730543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:38.730501    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:49.534195 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.534160    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"]
Jul 29 01:52:49.534729 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.534435    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr" podUID="c12b7d16-a24a-4e0f-9b0b-1117f887c677" containerName="maas-api" containerID="cri-o://21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c" gracePeriod=30
Jul 29 01:52:49.558724 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.558674    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"]
Jul 29 01:52:49.560587 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.560561    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-da474e-29754825-77rnj"]
Jul 29 01:52:49.562332 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.562312    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:49.600152 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.600126    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"]
Jul 29 01:52:49.600370 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.600339    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42" podUID="723be36f-4045-4762-8571-756bbc2d5c0f" containerName="payload-pre-processing" containerID="cri-o://fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c" gracePeriod=30
Jul 29 01:52:49.633937 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.633908    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:49.734148 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.734085    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:49.804131 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.803999    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:49.871712 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.871688    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:52:49.894042 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:49.894020    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:52:50.019369 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019341    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhl6l\" (UniqueName: \"kubernetes.io/projected/723be36f-4045-4762-8571-756bbc2d5c0f-kube-api-access-dhl6l\") pod \"723be36f-4045-4762-8571-756bbc2d5c0f\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") "
Jul 29 01:52:50.019561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019377    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftdr7\" (UniqueName: \"kubernetes.io/projected/c12b7d16-a24a-4e0f-9b0b-1117f887c677-kube-api-access-ftdr7\") pod \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") "
Jul 29 01:52:50.019561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019401    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723be36f-4045-4762-8571-756bbc2d5c0f-plugins-config-volume\") pod \"723be36f-4045-4762-8571-756bbc2d5c0f\" (UID: \"723be36f-4045-4762-8571-756bbc2d5c0f\") "
Jul 29 01:52:50.019561 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019524    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12b7d16-a24a-4e0f-9b0b-1117f887c677-openshift-service-ca\") pod \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") "
Jul 29 01:52:50.019751 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019611    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12b7d16-a24a-4e0f-9b0b-1117f887c677-maas-api-tls\") pod \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\" (UID: \"c12b7d16-a24a-4e0f-9b0b-1117f887c677\") "
Jul 29 01:52:50.019823 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019785    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723be36f-4045-4762-8571-756bbc2d5c0f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "723be36f-4045-4762-8571-756bbc2d5c0f" (UID: "723be36f-4045-4762-8571-756bbc2d5c0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:50.019878 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019842    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c12b7d16-a24a-4e0f-9b0b-1117f887c677-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c12b7d16-a24a-4e0f-9b0b-1117f887c677" (UID: "c12b7d16-a24a-4e0f-9b0b-1117f887c677"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:52:50.019947 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019939    2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723be36f-4045-4762-8571-756bbc2d5c0f-plugins-config-volume\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:50.019996 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.019953    2555 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12b7d16-a24a-4e0f-9b0b-1117f887c677-openshift-service-ca\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:50.021512 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.021477    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c12b7d16-a24a-4e0f-9b0b-1117f887c677-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c12b7d16-a24a-4e0f-9b0b-1117f887c677" (UID: "c12b7d16-a24a-4e0f-9b0b-1117f887c677"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 01:52:50.021619 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.021540    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12b7d16-a24a-4e0f-9b0b-1117f887c677-kube-api-access-ftdr7" (OuterVolumeSpecName: "kube-api-access-ftdr7") pod "c12b7d16-a24a-4e0f-9b0b-1117f887c677" (UID: "c12b7d16-a24a-4e0f-9b0b-1117f887c677"). InnerVolumeSpecName "kube-api-access-ftdr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:50.021619 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.021565    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723be36f-4045-4762-8571-756bbc2d5c0f-kube-api-access-dhl6l" (OuterVolumeSpecName: "kube-api-access-dhl6l") pod "723be36f-4045-4762-8571-756bbc2d5c0f" (UID: "723be36f-4045-4762-8571-756bbc2d5c0f"). InnerVolumeSpecName "kube-api-access-dhl6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:52:50.119115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.119040    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:50.120386 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.120363    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhl6l\" (UniqueName: \"kubernetes.io/projected/723be36f-4045-4762-8571-756bbc2d5c0f-kube-api-access-dhl6l\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:50.120445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.120390    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftdr7\" (UniqueName: \"kubernetes.io/projected/c12b7d16-a24a-4e0f-9b0b-1117f887c677-kube-api-access-ftdr7\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:50.120445 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.120401    2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12b7d16-a24a-4e0f-9b0b-1117f887c677-maas-api-tls\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:52:50.347010 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.346977    2555 generic.go:356] "Generic (PLEG): container finished" podID="c12b7d16-a24a-4e0f-9b0b-1117f887c677" containerID="21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c" exitCode=0
Jul 29 01:52:50.347185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.347023    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"
Jul 29 01:52:50.347185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.347056    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr" event={"ID":"c12b7d16-a24a-4e0f-9b0b-1117f887c677","Type":"ContainerDied","Data":"21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c"}
Jul 29 01:52:50.347185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.347118    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr" event={"ID":"c12b7d16-a24a-4e0f-9b0b-1117f887c677","Type":"ContainerDied","Data":"4e000b74784fa86f25755e9ae292d98a8957af547045e8424e0c1047c1b0799d"}
Jul 29 01:52:50.347185 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.347150    2555 scope.go:117] "RemoveContainer" containerID="21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c"
Jul 29 01:52:50.348514 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.348486    2555 generic.go:356] "Generic (PLEG): container finished" podID="723be36f-4045-4762-8571-756bbc2d5c0f" containerID="fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c" exitCode=0
Jul 29 01:52:50.348639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.348517    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42" event={"ID":"723be36f-4045-4762-8571-756bbc2d5c0f","Type":"ContainerDied","Data":"fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c"}
Jul 29 01:52:50.348639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.348539    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"
Jul 29 01:52:50.348639 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.348580    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42" event={"ID":"723be36f-4045-4762-8571-756bbc2d5c0f","Type":"ContainerDied","Data":"1a785eaa0dc8bd171c287ebea4a72d05aab799732489dc4c7c43541913e2ecde"}
Jul 29 01:52:50.359224 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.359205    2555 scope.go:117] "RemoveContainer" containerID="21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c"
Jul 29 01:52:50.359619 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:50.359530    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c\": container with ID starting with 21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c not found: ID does not exist" containerID="21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c"
Jul 29 01:52:50.359619 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.359555    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c"} err="failed to get container status \"21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c\": rpc error: code = NotFound desc = could not find container \"21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c\": container with ID starting with 21ff79dea2613efd9ebb64aab65dfdb4ab7dbef424d7180bfd49e77a0f04e09c not found: ID does not exist"
Jul 29 01:52:50.359619 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.359573    2555 scope.go:117] "RemoveContainer" containerID="fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c"
Jul 29 01:52:50.370583 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.370542    2555 scope.go:117] "RemoveContainer" containerID="fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c"
Jul 29 01:52:50.371122 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:52:50.370973    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c\": container with ID starting with fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c not found: ID does not exist" containerID="fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c"
Jul 29 01:52:50.371211 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.371131    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c"} err="failed to get container status \"fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c\": rpc error: code = NotFound desc = could not find container \"fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c\": container with ID starting with fd74f7a81c9b45efd1408666715f639de344509325268400db44b2136560637c not found: ID does not exist"
Jul 29 01:52:50.378431 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.378407    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"]
Jul 29 01:52:50.387517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.387496    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-da474e-645c48dcc-jxxnr"]
Jul 29 01:52:50.399500 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.399474    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"]
Jul 29 01:52:50.403472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:50.403448    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-da474e-5d6d4657c6-x9j42"]
Jul 29 01:52:51.189557 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:51.189523    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12296d79-96a8-4a7c-b1f9-c445dd31a952" path="/var/lib/kubelet/pods/12296d79-96a8-4a7c-b1f9-c445dd31a952/volumes"
Jul 29 01:52:51.190066 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:51.189997    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723be36f-4045-4762-8571-756bbc2d5c0f" path="/var/lib/kubelet/pods/723be36f-4045-4762-8571-756bbc2d5c0f/volumes"
Jul 29 01:52:51.190455 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:51.190435    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c12b7d16-a24a-4e0f-9b0b-1117f887c677" path="/var/lib/kubelet/pods/c12b7d16-a24a-4e0f-9b0b-1117f887c677/volumes"
Jul 29 01:52:56.049043 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:56.048963    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:56.164183 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:56.164155    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:56.331377 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:56.331293    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"]
Jul 29 01:52:56.331612 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:56.331530    2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" containerID="cri-o://50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070" gracePeriod=30
Jul 29 01:52:56.348680 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:56.348651    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:56.581325 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:56.581268    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:52:57.198536 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:57.198498    2555 patch_prober.go:28] interesting pod/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 01:52:57.198935 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:52:57.198555    2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 01:53:00.146138 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:00.146101    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:53:00.247676 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:00.247638    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:53:01.589521 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.589497    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:53:01.623385 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623312    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-envoy\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623385 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623349    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-socket\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623403    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvwrn\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-kube-api-access-bvwrn\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623458    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-token\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623486    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-podinfo\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623532    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-data\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623586 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623570    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-certs\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623830 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623624    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-credential-socket\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623830 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623651    2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4ff32af-ea91-46eb-a954-a3f1397e1371-istiod-ca-cert\") pod \"a4ff32af-ea91-46eb-a954-a3f1397e1371\" (UID: \"a4ff32af-ea91-46eb-a954-a3f1397e1371\") "
Jul 29 01:53:01.623830 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623685    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:53:01.623984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623858    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:53:01.623984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.623897    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:53:01.624093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.624019    2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-certs\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.624093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.624039    2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-credential-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.624093 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.624053    2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-workload-socket\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.624247 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.624192    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4ff32af-ea91-46eb-a954-a3f1397e1371-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 01:53:01.624788 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.624752    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:53:01.625721 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.625697    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-kube-api-access-bvwrn" (OuterVolumeSpecName: "kube-api-access-bvwrn") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "kube-api-access-bvwrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:53:01.626017 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.625988    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 01:53:01.626114 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.626093    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 01:53:01.626151 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.626093    2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4ff32af-ea91-46eb-a954-a3f1397e1371" (UID: "a4ff32af-ea91-46eb-a954-a3f1397e1371"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 01:53:01.725559 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.725519    2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4ff32af-ea91-46eb-a954-a3f1397e1371-istiod-ca-cert\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.725559 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.725552    2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-envoy\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.725559 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.725561    2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvwrn\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-kube-api-access-bvwrn\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.725559 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.725571    2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-token\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.725835 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.725580    2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-podinfo\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:01.725835 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:01.725587    2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4ff32af-ea91-46eb-a954-a3f1397e1371-istio-data\") on node \"ip-10-0-139-6.ec2.internal\" DevicePath \"\""
Jul 29 01:53:02.129543 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.129510    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jngrx_2f97c401-139c-4bba-b887-372be01cad11/manager/0.log"
Jul 29 01:53:02.239547 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.239513    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d7k7x_54dc7351-13ba-44e3-81e9-e2af92bd6c6c/manager/0.log"
Jul 29 01:53:02.362975 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.362943    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d584db87-4pl5c_f7312171-669b-4145-b5f7-5eb924d8d14b/manager/0.log"
Jul 29 01:53:02.411296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.411195    2555 generic.go:356] "Generic (PLEG): container finished" podID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerID="50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070" exitCode=0
Jul 29 01:53:02.411296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.411236    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" event={"ID":"a4ff32af-ea91-46eb-a954-a3f1397e1371","Type":"ContainerDied","Data":"50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070"}
Jul 29 01:53:02.411296 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.411249    2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"
Jul 29 01:53:02.411510 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.411313    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw" event={"ID":"a4ff32af-ea91-46eb-a954-a3f1397e1371","Type":"ContainerDied","Data":"406c328856671d2bd045c5da556d8927f8c06eb84503b5d909d3ae6e893d7c9e"}
Jul 29 01:53:02.411510 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.411349    2555 scope.go:117] "RemoveContainer" containerID="50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070"
Jul 29 01:53:02.420549 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.420533    2555 scope.go:117] "RemoveContainer" containerID="e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"
Jul 29 01:53:02.428116 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.428099    2555 scope.go:117] "RemoveContainer" containerID="50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070"
Jul 29 01:53:02.428389 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:53:02.428363    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070\": container with ID starting with 50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070 not found: ID does not exist" containerID="50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070"
Jul 29 01:53:02.428449 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.428400    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070"} err="failed to get container status \"50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070\": rpc error: code = NotFound desc = could not find container \"50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070\": container with ID starting with 50163b48349ecbb5f219a9cb597fd2e27d716198fd5e5c0136e994d4c447a070 not found: ID does not exist"
Jul 29 01:53:02.428449 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.428420    2555 scope.go:117] "RemoveContainer" containerID="e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"
Jul 29 01:53:02.428678 ip-10-0-139-6 kubenswrapper[2555]: E0729 01:53:02.428658    2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d\": container with ID starting with e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d not found: ID does not exist" containerID="e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"
Jul 29 01:53:02.428723 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.428684    2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d"} err="failed to get container status \"e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d\": rpc error: code = NotFound desc = could not find container \"e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d\": container with ID starting with e6aa917d71a254219f3085da76975df789569ff65d16314187dddd4c735a302d not found: ID does not exist"
Jul 29 01:53:02.434525 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.434504    2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"]
Jul 29 01:53:02.438127 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.438108    2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-da474e-openshift-default-6665cdc5b4-tgtqw"]
Jul 29 01:53:02.469324 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.469305    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jjw7d_88c84fa1-9280-4c6c-9836-a25983dc6586/server/0.log"
Jul 29 01:53:02.580102 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.580063    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h7xtb_01814704-c87c-4bbd-9b47-9d51634bf7fb/manager/0.log"
Jul 29 01:53:02.813458 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.813426    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-jf2w9_77d5b793-e5e3-4bf6-9571-ba6949991084/copy-manifests/0.log"
Jul 29 01:53:02.825482 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:02.825445    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-jf2w9_77d5b793-e5e3-4bf6-9571-ba6949991084/manager/0.log"
Jul 29 01:53:03.189517 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:03.189480    2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" path="/var/lib/kubelet/pods/a4ff32af-ea91-46eb-a954-a3f1397e1371/volumes"
Jul 29 01:53:03.271324 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:03.271288    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-dfd9875cc-7rgsp_470f09ae-2e69-4896-8148-0000a8858fbb/maas-api/0.log"
Jul 29 01:53:03.377196 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:03.377163    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754795-6smqj_af8eeec9-e4ea-45ba-83d6-128500bf86f5/cleanup/0.log"
Jul 29 01:53:03.698039 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:03.698003    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7lvmc_40659dba-adc5-4d4f-bc5a-536e27bec534/postgres/0.log"
Jul 29 01:53:05.034060 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:05.034031    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mm9wj_ce1cce01-bb9f-468d-b956-a81197604262/manager/0.log"
Jul 29 01:53:05.143324 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:05.143290    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h6ls4_0fc625e9-1b6e-443c-b0ee-9844adbdcd1b/manager/0.log"
Jul 29 01:53:05.361004 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:05.360929    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-s8hlq_0272b4f0-97aa-4f0a-baf4-b5b533feddb9/registry-server/0.log"
Jul 29 01:53:05.472888 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:05.472846    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vkbcv_74eb6dd2-6d54-4b91-8da7-046dcb5784df/manager/0.log"
Jul 29 01:53:05.695861 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:05.695824    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7vhrt_b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7/manager/0.log"
Jul 29 01:53:06.369407 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:06.369376    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-7p9fx_f8258603-0360-4bd2-95b2-e7474d23d308/kube-auth-proxy/0.log"
Jul 29 01:53:06.607391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:06.607359    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8gzhg_d418b254-d302-4d09-8c9b-8abc24d6ec18/payload-pre-processing/0.log"
Jul 29 01:53:06.727382 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:06.727350    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-z8k9t_83855546-55fb-4f87-948b-086b183adc64/payload-processing/0.log"
Jul 29 01:53:07.384081 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:07.384056    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-p44wj_fa6ab3e2-9490-4053-8d9b-4d0509b3b786/main/0.log"
Jul 29 01:53:07.599579 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:07.599548    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5rr86_73ec613f-7f12-460f-8584-9c39f969a8e1/main/0.log"
Jul 29 01:53:12.313002 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:12.312956    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:53:13.493536 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493500    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4fd22/must-gather-7tq2k"]
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493864    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy"
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493874    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy"
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493882    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="717b7270-61a1-4a98-b86d-aaf51c38fd8e" containerName="payload-pre-processing"
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493887    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="717b7270-61a1-4a98-b86d-aaf51c38fd8e" containerName="payload-pre-processing"
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493898    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12b7d16-a24a-4e0f-9b0b-1117f887c677" containerName="maas-api"
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493903    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12b7d16-a24a-4e0f-9b0b-1117f887c677" containerName="maas-api"
Jul 29 01:53:13.493909 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493909    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723be36f-4045-4762-8571-756bbc2d5c0f" containerName="payload-pre-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493915    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="723be36f-4045-4762-8571-756bbc2d5c0f" containerName="payload-pre-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493921    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493926    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493934    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35ddf2a8-f8f6-4767-8903-2aa696b091db" containerName="payload-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493941    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="35ddf2a8-f8f6-4767-8903-2aa696b091db" containerName="payload-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493955    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493961    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493966    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493971    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493981    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493986    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.493996    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494001    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494007    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494012    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494062    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12b7d16-a24a-4e0f-9b0b-1117f887c677" containerName="maas-api"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494070    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494076    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494081    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="35ddf2a8-f8f6-4767-8903-2aa696b091db" containerName="payload-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494086    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494092    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494099    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="723be36f-4045-4762-8571-756bbc2d5c0f" containerName="payload-pre-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494104    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="717b7270-61a1-4a98-b86d-aaf51c38fd8e" containerName="payload-pre-processing"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494108    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="64394b9d-5a6b-4d60-9522-4b465b15549e" containerName="istio-proxy"
Jul 29 01:53:13.494137 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494113    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f721afc5-0487-4360-9d01-7dcf8378211d" containerName="istio-proxy"
Jul 29 01:53:13.494895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494164    2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy"
Jul 29 01:53:13.494895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494169    2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy"
Jul 29 01:53:13.494895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494230    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ff32af-ea91-46eb-a954-a3f1397e1371" containerName="istio-proxy"
Jul 29 01:53:13.494895 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.494239    2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="422058f1-2f04-4672-b5a4-e6639c779b95" containerName="istio-proxy"
Jul 29 01:53:13.499123 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.499099    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.502124 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.502103    2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4fd22\"/\"default-dockercfg-8vnr7\""
Jul 29 01:53:13.502124 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.502119    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4fd22\"/\"openshift-service-ca.crt\""
Jul 29 01:53:13.503374 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.503355    2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4fd22\"/\"kube-root-ca.crt\""
Jul 29 01:53:13.507115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.507092    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4fd22/must-gather-7tq2k"]
Jul 29 01:53:13.508306 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.508271    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:53:13.611526 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.611486    2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log" limit=1024
Jul 29 01:53:13.642581 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.642552    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8e4cb14-6d4f-4b71-b674-42204ff1d852-must-gather-output\") pod \"must-gather-7tq2k\" (UID: \"e8e4cb14-6d4f-4b71-b674-42204ff1d852\") " pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.642715 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.642621    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rfds\" (UniqueName: \"kubernetes.io/projected/e8e4cb14-6d4f-4b71-b674-42204ff1d852-kube-api-access-7rfds\") pod \"must-gather-7tq2k\" (UID: \"e8e4cb14-6d4f-4b71-b674-42204ff1d852\") " pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.743721 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.743640    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8e4cb14-6d4f-4b71-b674-42204ff1d852-must-gather-output\") pod \"must-gather-7tq2k\" (UID: \"e8e4cb14-6d4f-4b71-b674-42204ff1d852\") " pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.743879 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.743723    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rfds\" (UniqueName: \"kubernetes.io/projected/e8e4cb14-6d4f-4b71-b674-42204ff1d852-kube-api-access-7rfds\") pod \"must-gather-7tq2k\" (UID: \"e8e4cb14-6d4f-4b71-b674-42204ff1d852\") " pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.744041 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.744017    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8e4cb14-6d4f-4b71-b674-42204ff1d852-must-gather-output\") pod \"must-gather-7tq2k\" (UID: \"e8e4cb14-6d4f-4b71-b674-42204ff1d852\") " pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.754180 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.754156    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rfds\" (UniqueName: \"kubernetes.io/projected/e8e4cb14-6d4f-4b71-b674-42204ff1d852-kube-api-access-7rfds\") pod \"must-gather-7tq2k\" (UID: \"e8e4cb14-6d4f-4b71-b674-42204ff1d852\") " pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.808906 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.808872    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4fd22/must-gather-7tq2k"
Jul 29 01:53:13.938065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.938037    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4fd22/must-gather-7tq2k"]
Jul 29 01:53:13.940648 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:53:13.940617    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8e4cb14_6d4f_4b71_b674_42204ff1d852.slice/crio-b1974b9ab46fb42c1cd18ca06afcc1f60a8e3f0ff60e3892c1cd15cd83344f4c WatchSource:0}: Error finding container b1974b9ab46fb42c1cd18ca06afcc1f60a8e3f0ff60e3892c1cd15cd83344f4c: Status 404 returned error can't find the container with id b1974b9ab46fb42c1cd18ca06afcc1f60a8e3f0ff60e3892c1cd15cd83344f4c
Jul 29 01:53:13.942703 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:13.942688    2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 01:53:14.477105 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:14.477058    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/must-gather-7tq2k" event={"ID":"e8e4cb14-6d4f-4b71-b674-42204ff1d852","Type":"ContainerStarted","Data":"b1974b9ab46fb42c1cd18ca06afcc1f60a8e3f0ff60e3892c1cd15cd83344f4c"}
Jul 29 01:53:15.487595 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:15.487558    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/must-gather-7tq2k" event={"ID":"e8e4cb14-6d4f-4b71-b674-42204ff1d852","Type":"ContainerStarted","Data":"5f40333d282a8100bffbc635f8399fc2c5241d8aaf81e357ea41f7246072af2b"}
Jul 29 01:53:15.488044 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:15.487608    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/must-gather-7tq2k" event={"ID":"e8e4cb14-6d4f-4b71-b674-42204ff1d852","Type":"ContainerStarted","Data":"d7de809ad598bbc6e2f7106059c6ad59d6cb5b4bbf94a691ab8585ada181160d"}
Jul 29 01:53:15.504418 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:15.504366    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4fd22/must-gather-7tq2k" podStartSLOduration=1.599744833 podStartE2EDuration="2.504349839s" totalImagesPullingTime="904.605006ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 01:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:53:15.50281103 +0000 UTC m=+3524.833460636" watchObservedRunningTime="2026-07-29 01:53:15.504349839 +0000 UTC m=+3524.834999454"
Jul 29 01:53:16.389262 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:16.389229    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dsgtn_be4f8050-958b-471a-8bb9-765583b21e3d/global-pull-secret-syncer/0.log"
Jul 29 01:53:16.487984 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:16.487951    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8lx2p_36158dc8-6ff8-4612-be4a-fccae1ee50f6/konnectivity-agent/0.log"
Jul 29 01:53:16.648483 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:16.648394    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-6.ec2.internal_df7a3dfe02b1862a6c7a987e8d3facb8/haproxy/0.log"
Jul 29 01:53:22.848582 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:22.848487    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mm9wj_ce1cce01-bb9f-468d-b956-a81197604262/manager/0.log"
Jul 29 01:53:22.884719 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:22.884608    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h6ls4_0fc625e9-1b6e-443c-b0ee-9844adbdcd1b/manager/0.log"
Jul 29 01:53:22.972098 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:22.972059    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-s8hlq_0272b4f0-97aa-4f0a-baf4-b5b533feddb9/registry-server/0.log"
Jul 29 01:53:23.158294 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:23.157883    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-vkbcv_74eb6dd2-6d54-4b91-8da7-046dcb5784df/manager/0.log"
Jul 29 01:53:23.295065 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:23.292211    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7vhrt_b7eaf57c-d80c-46fc-bc3f-7a61cc409bc7/manager/0.log"
Jul 29 01:53:24.970946 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:24.970910    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-b8mvr_e4c5caf5-64c2-4de7-a417-bf0497acbe98/kube-state-metrics/0.log"
Jul 29 01:53:24.990932 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:24.990900    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-b8mvr_e4c5caf5-64c2-4de7-a417-bf0497acbe98/kube-rbac-proxy-main/0.log"
Jul 29 01:53:25.014155 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.014122    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-b8mvr_e4c5caf5-64c2-4de7-a417-bf0497acbe98/kube-rbac-proxy-self/0.log"
Jul 29 01:53:25.037810 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.037776    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7d9b786d7f-6vtbt_aef8e36c-d565-4672-9210-99ec6d3cfffe/metrics-server/0.log"
Jul 29 01:53:25.062530 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.062495    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-5t97k_a117de77-4eed-4c31-83f0-d533610d2a71/monitoring-plugin/0.log"
Jul 29 01:53:25.089199 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.089168    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9pgl2_b6c0d77a-5903-4505-b25d-b0d1fcdae4c2/node-exporter/0.log"
Jul 29 01:53:25.109025 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.108994    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9pgl2_b6c0d77a-5903-4505-b25d-b0d1fcdae4c2/kube-rbac-proxy/0.log"
Jul 29 01:53:25.128556 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.128527    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9pgl2_b6c0d77a-5903-4505-b25d-b0d1fcdae4c2/init-textfile/0.log"
Jul 29 01:53:25.384653 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.384572    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/prometheus/0.log"
Jul 29 01:53:25.405662 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.405627    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/config-reloader/0.log"
Jul 29 01:53:25.426518 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.426481    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/thanos-sidecar/0.log"
Jul 29 01:53:25.445630 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.445600    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/kube-rbac-proxy-web/0.log"
Jul 29 01:53:25.465874 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.465842    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/kube-rbac-proxy/0.log"
Jul 29 01:53:25.487571 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.487536    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/kube-rbac-proxy-thanos/0.log"
Jul 29 01:53:25.508568 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:25.508536    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_88603ea8-5569-4040-afdc-66fd3a79a846/init-config-reloader/0.log"
Jul 29 01:53:26.140697 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.140659    2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"]
Jul 29 01:53:26.148397 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.148367    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.155428 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.155402    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"]
Jul 29 01:53:26.282695 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.282644    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-lib-modules\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.282996 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.282963    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-podres\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.283144 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.283129    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-sys\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.283391 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.283359    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59xtn\" (UniqueName: \"kubernetes.io/projected/b8440bfb-e90e-4beb-91e5-c65c15f369db-kube-api-access-59xtn\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.283559 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.283537    2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-proc\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.384524 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.384487    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59xtn\" (UniqueName: \"kubernetes.io/projected/b8440bfb-e90e-4beb-91e5-c65c15f369db-kube-api-access-59xtn\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.385197 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385171    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-proc\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.385533 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385517    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-lib-modules\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.385798 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385784    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-podres\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.386009 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385996    2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-sys\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.386248 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385431    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-proc\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.386396 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385956    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-podres\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.386491 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.385730    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-lib-modules\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.386575 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.386123    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8440bfb-e90e-4beb-91e5-c65c15f369db-sys\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.393348 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.393251    2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59xtn\" (UniqueName: \"kubernetes.io/projected/b8440bfb-e90e-4beb-91e5-c65c15f369db-kube-api-access-59xtn\") pod \"perf-node-gather-daemonset-l9l4g\" (UID: \"b8440bfb-e90e-4beb-91e5-c65c15f369db\") " pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.463997 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.463932    2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:26.648983 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:26.648876    2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"]
Jul 29 01:53:26.654931 ip-10-0-139-6 kubenswrapper[2555]: W0729 01:53:26.654899    2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb8440bfb_e90e_4beb_91e5_c65c15f369db.slice/crio-e59d60bcd025dd1fd35e9e48aee087670b4cfeecbd230de2b619e00547bca9e2 WatchSource:0}: Error finding container e59d60bcd025dd1fd35e9e48aee087670b4cfeecbd230de2b619e00547bca9e2: Status 404 returned error can't find the container with id e59d60bcd025dd1fd35e9e48aee087670b4cfeecbd230de2b619e00547bca9e2
Jul 29 01:53:27.574603 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:27.574559    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g" event={"ID":"b8440bfb-e90e-4beb-91e5-c65c15f369db","Type":"ContainerStarted","Data":"a8cadbb21ba6acb1adf5a99791810adc79d3d29d9b03aeda667a28a37af496a1"}
Jul 29 01:53:27.575084 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:27.574631    2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g" event={"ID":"b8440bfb-e90e-4beb-91e5-c65c15f369db","Type":"ContainerStarted","Data":"e59d60bcd025dd1fd35e9e48aee087670b4cfeecbd230de2b619e00547bca9e2"}
Jul 29 01:53:27.575084 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:27.574687    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:27.592339 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:27.592247    2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g" podStartSLOduration=1.5922284389999999 podStartE2EDuration="1.592228439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 01:53:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 01:53:27.590295973 +0000 UTC m=+3536.920945579" watchObservedRunningTime="2026-07-29 01:53:27.592228439 +0000 UTC m=+3536.922878046"
Jul 29 01:53:28.650226 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:28.650190    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-gxfx4_9f5f978a-057a-4967-9c93-d42238951f2c/volume-data-source-validator/0.log"
Jul 29 01:53:29.836894 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:29.836861    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hpq5j_1b15c048-7e7f-4ca3-8b69-8f23cab9d488/dns/0.log"
Jul 29 01:53:29.859115 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:29.859085    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hpq5j_1b15c048-7e7f-4ca3-8b69-8f23cab9d488/kube-rbac-proxy/0.log"
Jul 29 01:53:29.936989 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:29.936963    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w44jh_cdc09b7a-c22b-4e09-b828-1b7a4ef8c289/dns-node-resolver/0.log"
Jul 29 01:53:30.469536 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:30.469506    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b4468786b-df8lr_e1db99fd-6618-4071-9c40-c2c1e5dd47ef/registry/0.log"
Jul 29 01:53:30.542950 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:30.542912    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tsn5b_05209071-ba00-4003-8c93-5b47ca7c3d8f/node-ca/0.log"
Jul 29 01:53:31.805073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:31.805033    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-7p9fx_f8258603-0360-4bd2-95b2-e7474d23d308/kube-auth-proxy/0.log"
Jul 29 01:53:31.931241 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:31.931201    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8gzhg_d418b254-d302-4d09-8c9b-8abc24d6ec18/payload-pre-processing/0.log"
Jul 29 01:53:31.964019 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:31.963986    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-z8k9t_83855546-55fb-4f87-948b-086b183adc64/payload-processing/0.log"
Jul 29 01:53:32.818472 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:32.818430    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k2bfw_b249760f-1f36-4430-ab58-22f44d91f7a9/serve-healthcheck-canary/0.log"
Jul 29 01:53:33.460681 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:33.460648    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hbpj8_46fbc0f2-836f-43aa-8906-a91ec6c61b27/kube-rbac-proxy/0.log"
Jul 29 01:53:33.488845 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:33.488800    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hbpj8_46fbc0f2-836f-43aa-8906-a91ec6c61b27/exporter/0.log"
Jul 29 01:53:33.513073 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:33.513033    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hbpj8_46fbc0f2-836f-43aa-8906-a91ec6c61b27/extractor/0.log"
Jul 29 01:53:33.590989 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:33.590956    2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4fd22/perf-node-gather-daemonset-l9l4g"
Jul 29 01:53:34.048886 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:34.048851    2555 scope.go:117] "RemoveContainer" containerID="4f2c14f1e25fba12464d8d7682ea5e49061f65645bcef8ef781d0b6cd3ba9c9b"
Jul 29 01:53:36.046837 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.046755    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jngrx_2f97c401-139c-4bba-b887-372be01cad11/manager/0.log"
Jul 29 01:53:36.357601 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.357567    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d7k7x_54dc7351-13ba-44e3-81e9-e2af92bd6c6c/manager/0.log"
Jul 29 01:53:36.533606 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.533573    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d584db87-4pl5c_f7312171-669b-4145-b5f7-5eb924d8d14b/manager/0.log"
Jul 29 01:53:36.554580 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.554502    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jjw7d_88c84fa1-9280-4c6c-9836-a25983dc6586/server/0.log"
Jul 29 01:53:36.650417 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.648061    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-h7xtb_01814704-c87c-4bbd-9b47-9d51634bf7fb/manager/0.log"
Jul 29 01:53:36.800329 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.800267    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-jf2w9_77d5b793-e5e3-4bf6-9571-ba6949991084/manager/0.log"
Jul 29 01:53:36.816054 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:36.815975    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-795cff746b-jf2w9_77d5b793-e5e3-4bf6-9571-ba6949991084/copy-manifests/0.log"
Jul 29 01:53:37.088794 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:37.088723    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-7p9fx_f8258603-0360-4bd2-95b2-e7474d23d308/kube-auth-proxy/0.log"
Jul 29 01:53:37.198457 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:37.198425    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8gzhg_d418b254-d302-4d09-8c9b-8abc24d6ec18/payload-pre-processing/0.log"
Jul 29 01:53:37.223452 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:37.223396    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-z8k9t_83855546-55fb-4f87-948b-086b183adc64/payload-processing/0.log"
Jul 29 01:53:38.946021 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:38.945991    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b44fbcd-7p9fx_f8258603-0360-4bd2-95b2-e7474d23d308/kube-auth-proxy/0.log"
Jul 29 01:53:39.049957 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:39.049924    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-8gzhg_d418b254-d302-4d09-8c9b-8abc24d6ec18/payload-pre-processing/0.log"
Jul 29 01:53:39.075991 ip-10-0-139-6 kubenswrapper[2555]: I0729 01:53:39.075955    2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-z8k9t_83855546-55fb-4f87-948b-086b183adc64/payload-processing/0.log"