Sep 11 18:10:30.246655 ip-10-0-140-142 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 18:10:30.653059 ip-10-0-140-142 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 18:10:30.653059 ip-10-0-140-142 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 18:10:30.653059 ip-10-0-140-142 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 18:10:30.653059 ip-10-0-140-142 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 18:10:30.653059 ip-10-0-140-142 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 18:10:30.653665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.653133    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 18:10:30.657391 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657361    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:10:30.657391 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657390    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657395    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657398    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657401    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657404    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657407    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657409    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657412    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657415    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657420    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657424    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657427    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657430    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657439    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657442    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657444    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657447    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657450    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657452    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:10:30.657455 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657455    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657457    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657460    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657463    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657466    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657469    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657472    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657474    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657477    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657479    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657482    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657485    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657487    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657490    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657492    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657495    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657497    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657499    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657502    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657504    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:10:30.657900 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657507    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657509    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657511    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657514    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657516    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657519    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657521    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657531    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657535    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657538    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657540    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657543    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657546    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657549    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657552    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657555    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657558    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657560    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657563    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:10:30.658507 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657566    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657569    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657571    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657574    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657576    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657579    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657581    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657584    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657587    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657589    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657592    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657594    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657596    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657599    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657601    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657604    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657606    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657609    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657611    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657613    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:10:30.659092 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657617    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657622    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657625    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657628    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657630    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657633    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657636    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657640    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657642    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657645    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657648    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657651    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657656    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657659    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657661    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657664    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657667    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657669    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657672    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:10:30.659829 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.657674    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658426    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658433    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658437    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658439    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658442    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658445    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658447    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658450    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658452    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658455    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658458    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658460    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658463    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658465    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658468    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658472    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658476    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658479    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:10:30.660306 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658482    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658486    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658488    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658491    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658494    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658497    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658499    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658502    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658504    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658507    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658510    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658513    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658516    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658518    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658522    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658525    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658528    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658530    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658533    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658536    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:10:30.660823 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658538    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658541    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658544    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658546    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658549    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658551    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658554    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658556    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658559    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658564    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658567    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658570    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658573    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658576    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658579    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658581    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658584    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658586    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658589    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658591    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:10:30.661348 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658594    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658596    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658599    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658601    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658604    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658606    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658609    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658611    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658614    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658617    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658620    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658622    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658625    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658627    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658631    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658635    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658638    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658641    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658644    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:10:30.661865 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658739    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658758    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658764    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658770    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658775    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658780    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658785    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658790    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658795    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658799    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658805    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658889    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658908    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658915    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658921    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658927    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658933    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658938    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658942    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658947    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:10:30.662330 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658952    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.658956    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660116    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660125    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660128    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660132    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660135    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660138    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660142    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660145    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660148    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660150    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660153    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660155    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660158    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660161    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660163    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660166    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660168    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660171    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:10:30.662856 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660173    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660176    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660179    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660181    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660183    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660186    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660188    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660197    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660200    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660203    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660205    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660209    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660213    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660215    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660218    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660221    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660223    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660226    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660229    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:10:30.663334 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660232    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660234    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660237    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660240    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660243    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660245    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660248    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660250    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660253    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660256    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660259    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660261    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660264    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660267    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660270    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660272    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660275    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660277    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660280    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:10:30.663810 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660282    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660285    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660288    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660290    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660293    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660295    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660298    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660301    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660303    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660306    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660308    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660311    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660314    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660316    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660319    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660322    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660325    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660327    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660330    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660332    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:10:30.664257 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660335    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660337    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660340    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660343    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660345    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660347    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660350    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660353    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660356    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660358    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660361    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660385    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660388    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660391    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660393    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660396    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660399    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660403    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660407    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:10:30.664760 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660410    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660415    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660418    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.660421    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660491    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660536    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660543    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660548    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660552    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660555    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660559    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660563    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660566    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660569    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660573    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660576    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660579    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660582    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660585    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660588    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660590    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660593    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660599    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 11 18:10:30.665221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660601    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660604    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660607    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660611    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660615    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660618    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660621    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660624    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660628    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660631    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660634    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660637    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660640    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660644    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660647    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660650    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660653    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660656    2563 flags.go:64] FLAG: --enable-server="true"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660659    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660663    2563 flags.go:64] FLAG: --event-burst="100"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660666    2563 flags.go:64] FLAG: --event-qps="50"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660669    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660672    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660675    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 11 18:10:30.665809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660678    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660681    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660684    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660687    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660690    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660693    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660696    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660699    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660702    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660705    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660708    2563 flags.go:64] FLAG: --feature-gates=""
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660716    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660719    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660722    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660726    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660729    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660731    2563 flags.go:64] FLAG: --help="false"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660735    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-140-142.ec2.internal"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660739    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660742    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660747    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660751    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660757    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660759    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 18:10:30.666651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660762    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660765    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660768    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660771    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660774    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660776    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660779    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660782    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660785    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660788    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660791    2563 flags.go:64] FLAG: --lock-file=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660794    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660797    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660800    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660806    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660809    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660811    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660814    2563 flags.go:64] FLAG: --logging-format="text"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660817    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660820    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660823    2563 flags.go:64] FLAG: --manifest-url=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660827    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660831    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660834    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660838    2563 flags.go:64] FLAG: --max-pods="110"
Sep 11 18:10:30.667774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660841    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660845    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660848    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660850    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660853    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660857    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660860    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660867    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660870    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660873    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660876    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660879    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660884    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660887    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660890    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660893    2563 flags.go:64] FLAG: --port="10250"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660896    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660899    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fd592488cab7ca91"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660904    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660907    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660910    2563 flags.go:64] FLAG: --register-node="true"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660913    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660916    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660919    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 11 18:10:30.668844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660923    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660926    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660933    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660936    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660939    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660942    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660945    2563 flags.go:64] FLAG: --runonce="false"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660948    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660951    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660954    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660958    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660960    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660963    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660969    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660972    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660975    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660978    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660981    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660984    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660987    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660990    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660993    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.660998    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661001    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661004    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 18:10:30.669923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661008    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661013    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661015    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661018    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661021    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661024    2563 flags.go:64] FLAG: --v="2"
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661028    2563 flags.go:64] FLAG: --version="false"
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661032    2563 flags.go:64] FLAG: --vmodule=""
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661035    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661038    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661168    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661172    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661175    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661178    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661181    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661184    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661186    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661189    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661192    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661195    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661198    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661201    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:10:30.670769 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661204    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661206    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661211    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661215    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661217    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661220    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661223    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661226    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661228    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661231    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661233    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661238    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661241    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661243    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661246    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661250    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661253    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661256    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661259    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661262    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:10:30.671772 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661264    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661267    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661269    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661272    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661275    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661278    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661280    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661283    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661285    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661289    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661292    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661296    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661300    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661304    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661308    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661312    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661314    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661317    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661320    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661322    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:10:30.672518 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661325    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661328    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661330    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661334    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661337    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661340    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661342    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661345    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661348    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661350    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661353    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661356    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661359    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661361    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661377    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661380    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661383    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661385    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661388    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:10:30.673179 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661391    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661394    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661396    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661401    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661403    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661406    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661409    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661411    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661414    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661416    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661419    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661421    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661424    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661427    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661429    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661432    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661436    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661439    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661441    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661444    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:10:30.673665 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661447    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661449    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661452    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661455    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661459    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661463    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661466    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.661469    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.661474    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.668853    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.668873    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.668895    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.668904    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.668989    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:10:30.674203 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.668997    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669004    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669010    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669015    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669020    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669025    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669029    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669034    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669038    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669042    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669046    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669050    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669055    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669060    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669064    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669068    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669073    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669077    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669081    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669086    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:10:30.674574 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669091    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669095    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669099    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669103    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669108    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669112    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669116    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669121    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669126    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669131    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669136    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669140    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669144    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669148    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669152    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669156    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669161    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669165    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669170    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669174    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:10:30.675135 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669178    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669182    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669186    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669191    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669195    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669199    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669203    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669207    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669211    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669215    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669220    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669225    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669229    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669233    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669238    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669242    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669246    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669250    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669254    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669258    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:10:30.675675 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669263    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669272    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669276    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669280    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669285    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669291    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669296    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669300    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669305    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669309    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669316    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669322    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669327    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669332    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669337    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669342    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669347    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669352    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669356    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:10:30.676210 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669361    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669384    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669388    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669392    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669396    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669400    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669404    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669408    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669412    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669417    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669422    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669426    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669430    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669434    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669438    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669448    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669453    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669457    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:10:30.677142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669463    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.669472    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669650    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669658    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669663    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669667    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669672    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669676    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669680    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669684    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669688    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669693    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669697    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669701    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669705    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 18:10:30.677654 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669709    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669713    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669717    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669721    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669725    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669730    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669734    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669738    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669741    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669748    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669754    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669760    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669764    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669768    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669773    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669778    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669782    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669786    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669790    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 18:10:30.678027 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669794    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669799    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669803    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669807    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669812    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669815    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669820    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669824    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669828    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669832    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669836    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669840    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669845    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669851    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669856    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669861    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669865    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669869    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669873    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669877    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 18:10:30.678521 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669881    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669886    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669891    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669895    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669899    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669903    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669907    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669912    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669916    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669920    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669924    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669928    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669932    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669936    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669939    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669943    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669947    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669951    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669955    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669960    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 18:10:30.679242 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669964    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669968    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669972    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669976    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669980    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669984    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669988    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669992    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.669997    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670003    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670008    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670013    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670017    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670022    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670027    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670031    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670035    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670039    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670044    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 18:10:30.680113 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670048    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670052    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670056    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670061    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670066    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670070    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670074    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:30.670078    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.670086    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.670267    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.672840    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.673866    2563 server.go:1019] "Starting client certificate rotation"
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.673967    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 18:10:30.680626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.674009    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 18:10:30.680972 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.676699    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 18:10:30.680972 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.680012    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 18:10:30.691246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.691120    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 11 18:10:30.695774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.695759    2563 log.go:25] "Validated CRI v1 image API"
Sep 11 18:10:30.697489 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.697473    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 18:10:30.699826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.699805    2563 fs.go:135] Filesystem UUIDs: map[5b9ee2d1-7136-4394-8e72-a52351115758:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 906ad45d-9cc5-47d0-8095-c2863836305e:/dev/nvme0n1p4]
Sep 11 18:10:30.699892 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.699824    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 18:10:30.706018 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.705914    2563 manager.go:216] Machine: {Timestamp:2026-09-11 18:10:30.704625511 +0000 UTC m=+0.343719320 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101184 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fe755c315d5ca12a5aedd5be57c89 SystemUUID:ec2fe755-c315-d5ca-12a5-aedd5be57c89 BootID:0561ef44-b036-4860-a51a-d1e86788f022 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f8:52:2b:dd:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f8:52:2b:dd:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:fc:d9:c2:ee:83 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 18:10:30.706018 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.706016    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 18:10:30.706121 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.706082    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 18:10:30.709661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.709632    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 18:10:30.709794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.709662    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 18:10:30.710401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.710391    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 18:10:30.710432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.710404    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 18:10:30.710432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.710411    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 18:10:30.710788 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.710772    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 18:10:30.711044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.711031    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 18:10:30.711842 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.711832    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 18:10:30.715717 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715705    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 18:10:30.715832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715823    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 18:10:30.715866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715852    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 18:10:30.715866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715861    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 18:10:30.715919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715876    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 18:10:30.715919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715888    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 11 18:10:30.715919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.715907    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 18:10:30.716227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.716216    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 18:10:30.716267 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.716237    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 18:10:30.718360 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.718344    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 18:10:30.720717 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.720702    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 18:10:30.720774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.720733    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 18:10:30.722497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722484    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 18:10:30.722497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722500    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722506    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722512    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722520    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722528    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722548    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722553    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722559    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 18:10:30.722571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722564    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 18:10:30.722803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722580    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 18:10:30.722803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.722586    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 18:10:30.724218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.724204    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 18:10:30.724218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.724215    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 18:10:30.725142 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.725116    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 18:10:30.725203 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.725150    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 18:10:30.727759 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.727746    2563 server.go:1268] "Started kubelet"
Sep 11 18:10:30.727871 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.727831    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 18:10:30.727926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.727890    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 18:10:30.727926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.727899    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 18:10:30.728522 ip-10-0-140-142 systemd[1]: Started Kubernetes Kubelet.
Sep 11 18:10:30.729553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.729231    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 18:10:30.731294 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.730216    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 18:10:30.734174 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.734142    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 18:10:30.735966 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.735943    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 18:10:30.736601 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.736582    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 18:10:30.736878 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.735917    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-142.ec2.internal.18d456a7d0557ea0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-142.ec2.internal,UID:ip-10-0-140-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-142.ec2.internal,},FirstTimestamp:2026-09-11 18:10:30.727720608 +0000 UTC m=+0.366814417,LastTimestamp:2026-09-11 18:10:30.727720608 +0000 UTC m=+0.366814417,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-142.ec2.internal,}"
Sep 11 18:10:30.737199 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737178    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 18:10:30.737518 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.737496    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:30.737830 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737800    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 18:10:30.737830 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737803    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 18:10:30.737830 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737831    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 18:10:30.737969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737872    2563 factory.go:55] Registering systemd factory
Sep 11 18:10:30.737969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737879    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 18:10:30.737969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737887    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 18:10:30.737969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.737888    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 11 18:10:30.738093 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.738080    2563 factory.go:153] Registering CRI-O factory
Sep 11 18:10:30.738128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.738095    2563 factory.go:223] Registration of the crio container factory successfully
Sep 11 18:10:30.738168 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.738150    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 18:10:30.738219 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.738169    2563 factory.go:103] Registering Raw factory
Sep 11 18:10:30.738219 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.738184    2563 manager.go:1196] Started watching for new ooms in manager
Sep 11 18:10:30.738594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.738577    2563 manager.go:349] Starting recovery of all containers
Sep 11 18:10:30.740146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.740124    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k2w9h"
Sep 11 18:10:30.744409 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.744383    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 18:10:30.745007 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.744963    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 18:10:30.746804 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.746786    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-k2w9h"
Sep 11 18:10:30.748860 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.748844    2563 manager.go:354] Recovery completed
Sep 11 18:10:30.753730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.753717    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.755489 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.755472    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.755572 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.755498    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.755572 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.755507    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.756024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756007    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 18:10:30.756024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756020    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.756024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756023    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 18:10:30.756218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756039    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 18:10:30.756694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756679    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.756771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756705    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.756771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.756715    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.757240 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.757228    2563 policy_none.go:49] "None policy: Start"
Sep 11 18:10:30.757290 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.757243    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 18:10:30.757290 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.757252    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 18:10:30.758077 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.758021    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-142.ec2.internal.18d456a7d1fd2b12  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-142.ec2.internal,UID:ip-10-0-140-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-142.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-142.ec2.internal,},FirstTimestamp:2026-09-11 18:10:30.755486482 +0000 UTC m=+0.394580291,LastTimestamp:2026-09-11 18:10:30.755486482 +0000 UTC m=+0.394580291,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-142.ec2.internal,}"
Sep 11 18:10:30.758353 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.758342    2563 policy_none.go:47] "Start"
Sep 11 18:10:30.788957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.788942    2563 manager.go:337] "Starting Device Plugin manager"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.788985    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.788997    2563 server.go:87] "Starting device plugin registration server"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.789300    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.789312    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.789452    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.789508    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.789517    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.789959    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 18:10:30.804193 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.790010    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:30.858896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.858871    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 18:10:30.859944 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.859927    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 18:10:30.860025 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.859982    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 18:10:30.860025 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.860006    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 18:10:30.860112 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.860044    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 18:10:30.862075 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.862057    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:10:30.889956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.889942    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.890706 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.890692    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.890772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.890719    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.890772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.890732    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.890772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.890751    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-142.ec2.internal"
Sep 11 18:10:30.899124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.899109    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-142.ec2.internal"
Sep 11 18:10:30.899193 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.899128    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-142.ec2.internal\": node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:30.924663 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.924608    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:30.960124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.960104    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"]
Sep 11 18:10:30.960185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.960161    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.961430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.961416    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.961495 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.961440    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.961495 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.961450    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.962579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.962567    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.962762 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.962749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:30.962797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.962777    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.963205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.963188    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.963264 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.963216    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.963264 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.963191    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.963264 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.963227    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.963264 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.963244    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.963264 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.963255    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.964400 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.964385    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:30.964454 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.964411    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 18:10:30.964999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.964986    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 18:10:30.965070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.965008    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 18:10:30.965070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:30.965020    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeHasSufficientPID"
Sep 11 18:10:30.993955 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.993936    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-142.ec2.internal\" not found" node="ip-10-0-140-142.ec2.internal"
Sep 11 18:10:30.998236 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:30.998222    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-142.ec2.internal\" not found" node="ip-10-0-140-142.ec2.internal"
Sep 11 18:10:31.025695 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:31.025679    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:31.126482 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:31.126464    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:31.139768 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:31.139751    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9add31ed36a5b5cac8ed65d2c77f17bf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-142.ec2.internal\" (UID: \"9add31ed36a5b5cac8ed65d2c77f17bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:31.139834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:31.139772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ff41c913f1d6f6987c1f137b7ff6119-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3ff41c913f1d6f6987c1f137b7ff6119\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:31.139834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:31.139793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ff41c913f1d6f6987c1f137b7ff6119-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3ff41c913f1d6f6987c1f137b7ff6119\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.273241 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.273224    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.286541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.286524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ff41c913f1d6f6987c1f137b7ff6119-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3ff41c913f1d6f6987c1f137b7ff6119\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.286584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.286547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ff41c913f1d6f6987c1f137b7ff6119-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3ff41c913f1d6f6987c1f137b7ff6119\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.286584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.286564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9add31ed36a5b5cac8ed65d2c77f17bf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-142.ec2.internal\" (UID: \"9add31ed36a5b5cac8ed65d2c77f17bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.286658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.286598    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9add31ed36a5b5cac8ed65d2c77f17bf-config\") pod \"kube-apiserver-proxy-ip-10-0-140-142.ec2.internal\" (UID: \"9add31ed36a5b5cac8ed65d2c77f17bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.286658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.286628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3ff41c913f1d6f6987c1f137b7ff6119-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3ff41c913f1d6f6987c1f137b7ff6119\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.286658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.286630    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ff41c913f1d6f6987c1f137b7ff6119-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal\" (UID: \"3ff41c913f1d6f6987c1f137b7ff6119\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.342665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.342642    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.347567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.347549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:32.374189 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.374165    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.474674 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.474622    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.575102 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.575071    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.675614 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.675596    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.720062 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.720044    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 18:10:32.720423 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.720166    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 18:10:32.776493 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.776448    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.797230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.797196    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 18:05:31 +0000 UTC" deadline="2028-03-11 09:47:44.911582823 +0000 UTC"
Sep 11 18:10:32.797230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.797227    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13119h37m12.114359198s"
Sep 11 18:10:32.803446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.803431    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:10:32.877005 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.876970    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.912478 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:32.912453    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff41c913f1d6f6987c1f137b7ff6119.slice/crio-99a7def03f44a4670fd5100d8ef564c1b1221de919fee6a0d6b578022feac8f1 WatchSource:0}: Error finding container 99a7def03f44a4670fd5100d8ef564c1b1221de919fee6a0d6b578022feac8f1: Status 404 returned error can't find the container with id 99a7def03f44a4670fd5100d8ef564c1b1221de919fee6a0d6b578022feac8f1
Sep 11 18:10:32.912888 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:32.912871    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9add31ed36a5b5cac8ed65d2c77f17bf.slice/crio-439715a12f4bcf75642a289bb2500b60464d1f0f53902ad5709c0ec1bc1ea4c1 WatchSource:0}: Error finding container 439715a12f4bcf75642a289bb2500b60464d1f0f53902ad5709c0ec1bc1ea4c1: Status 404 returned error can't find the container with id 439715a12f4bcf75642a289bb2500b60464d1f0f53902ad5709c0ec1bc1ea4c1
Sep 11 18:10:32.917428 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.917411    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:10:32.977764 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:32.977739    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-142.ec2.internal\" not found"
Sep 11 18:10:32.995165 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:32.995150    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:10:33.084001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.083959    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:33.096117 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.096100    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 18:10:33.097178 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.097164    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal"
Sep 11 18:10:33.107019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.107000    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 18:10:33.136334 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.136314    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:10:33.549579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.549510    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 18:10:33.763322 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.763293    2563 apiserver.go:52] "Watching apiserver"
Sep 11 18:10:33.771822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.771795    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 18:10:33.772248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.772210    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-szmgh","kube-system/global-pull-secret-syncer-6ntmn","kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx","openshift-dns/node-resolver-x46v2","openshift-multus/multus-additional-cni-plugins-4rdld","openshift-multus/network-metrics-daemon-qhmz2","openshift-network-diagnostics/network-check-target-m67bj","openshift-ovn-kubernetes/ovnkube-node-7bpkd","kube-system/konnectivity-agent-4vhkv","openshift-cluster-node-tuning-operator/tuned-wzld5","openshift-image-registry/node-ca-4ft7z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal","openshift-multus/multus-9b54z"]
Sep 11 18:10:33.773668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.773648    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.775150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.774772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.775866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.775817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.777152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.777135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.777879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.777862    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.777995 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.777863    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.779052 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.779036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.781132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.780747    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.781132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.780943    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mh8lm\""
Sep 11 18:10:33.782002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.781985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:33.782363 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.782333    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:33.782476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.782102    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 18:10:33.782568 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.782547    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 18:10:33.782707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.782066    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 18:10:33.782864 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.782785    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 18:10:33.782936 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.782496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.782998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.782220    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 18:10:33.783070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.783057    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 18:10:33.783203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.783182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.783459    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.783701    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.783807    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nqxnw\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784020    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784135    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784278    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9bt7x\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784419    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gbqvz\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784488    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zjdn5\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784498    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.784631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:33.784747 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.784689    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:33.785337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.785083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.786440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.786418    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.787837 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.787820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:33.787926 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.787881    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:33.789052 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.789036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.791182 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.791083    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.791728    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.791843    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.791962    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792067    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792124    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792311    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792321    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792444    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7k9mq\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792471    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792515    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792738    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 18:10:33.792988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792823    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w5vnr\""
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.792993    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nbp5d\""
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793175    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c3e22d68-3089-47b8-b9b1-1c81db256a73-agent-certs\") pod \"konnectivity-agent-4vhkv\" (UID: \"c3e22d68-3089-47b8-b9b1-1c81db256a73\") " pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cnibin\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-registration-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-slash\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-systemd\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovn-node-metrics-cert\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lllv\" (UniqueName: \"kubernetes.io/projected/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-kube-api-access-4lllv\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793398    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b656de9-f949-4ae4-b1a7-9ee77032b734-hosts-file\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793424    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-socket-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793478    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-sys-fs\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbfm4\" (UniqueName: \"kubernetes.io/projected/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-kube-api-access-wbfm4\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.793596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-run-netns\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-log-socket\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/117de80d-dc8d-43d8-b406-85c3a636ecb4-serviceca\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793666    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-device-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-etc-selinux\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-var-lib-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-ovn\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-node-log\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c74h\" (UniqueName: \"kubernetes.io/projected/05b6c004-f567-4008-9544-1fc839d96155-kube-api-access-7c74h\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-iptables-alerter-script\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-run-ovn-kubernetes\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.793998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c3e22d68-3089-47b8-b9b1-1c81db256a73-konnectivity-ca\") pod \"konnectivity-agent-4vhkv\" (UID: \"c3e22d68-3089-47b8-b9b1-1c81db256a73\") " pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8vg4\" (UniqueName: \"kubernetes.io/projected/117de80d-dc8d-43d8-b406-85c3a636ecb4-kube-api-access-m8vg4\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.794285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-host-slash\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrc2v\" (UniqueName: \"kubernetes.io/projected/df55e017-4726-49f9-8507-41222acd67f0-kube-api-access-zrc2v\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-cni-bin\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-cni-netd\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794204    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovnkube-script-lib\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b656de9-f949-4ae4-b1a7-9ee77032b734-tmp-dir\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-system-cni-dir\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794308    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-kubelet\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/117de80d-dc8d-43d8-b406-85c3a636ecb4-host\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovnkube-config\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln456\" (UniqueName: \"kubernetes.io/projected/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-kube-api-access-ln456\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.794997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-systemd-units\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.795723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-etc-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.795723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-env-overrides\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.795723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.794640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2x28\" (UniqueName: \"kubernetes.io/projected/3b656de9-f949-4ae4-b1a7-9ee77032b734-kube-api-access-t2x28\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.800427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.800336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 18:10:33.800757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.800717    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fmtn9\""
Sep 11 18:10:33.816009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.815983    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 18:10:33.847795 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.847774    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fkzrc"
Sep 11 18:10:33.854996 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.854977    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fkzrc"
Sep 11 18:10:33.885389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.885371    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 18:10:33.895396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-systemd-units\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-env-overrides\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cnibin\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.895501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xxwv\" (UniqueName: \"kubernetes.io/projected/06168f80-3164-4c7e-a316-f7449b680db9-kube-api-access-9xxwv\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.895501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-systemd-units\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cnibin\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.895653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-slash\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lllv\" (UniqueName: \"kubernetes.io/projected/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-kube-api-access-4lllv\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895582    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-run\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.895653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-cni-bin\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.895653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-etc-kubernetes\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-sys-fs\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-sys-fs\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895745    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06168f80-3164-4c7e-a316-f7449b680db9-multus-daemon-config\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-slash\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-ovn\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895870    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-env-overrides\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-ovn\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-node-log\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.895957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-sys\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.895968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-cni-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-node-log\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06168f80-3164-4c7e-a316-f7449b680db9-cni-binary-copy\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-socket-dir-parent\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-host-slash\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.896220    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896276    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-host-slash\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.896293    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:34.396274725 +0000 UTC m=+2.989475280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrc2v\" (UniqueName: \"kubernetes.io/projected/df55e017-4726-49f9-8507-41222acd67f0-kube-api-access-zrc2v\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-cni-bin\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-cni-bin\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-system-cni-dir\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.896446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/117de80d-dc8d-43d8-b406-85c3a636ecb4-host\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896574    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-system-cni-dir\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysconfig\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/117de80d-dc8d-43d8-b406-85c3a636ecb4-host\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f2089a3-9817-4c29-980b-f4018a7b145e-kubelet-config\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-kubelet\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-var-lib-kubelet\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-kubelet\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-cni-multus\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.896979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovnkube-config\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-cnibin\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-etc-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.897227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2x28\" (UniqueName: \"kubernetes.io/projected/3b656de9-f949-4ae4-b1a7-9ee77032b734-kube-api-access-t2x28\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c3e22d68-3089-47b8-b9b1-1c81db256a73-agent-certs\") pod \"konnectivity-agent-4vhkv\" (UID: \"c3e22d68-3089-47b8-b9b1-1c81db256a73\") " pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-system-cni-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-etc-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897198    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-conf-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-registration-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-systemd\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovn-node-metrics-cert\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b656de9-f949-4ae4-b1a7-9ee77032b734-hosts-file\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897354    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-run-systemd\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897374    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-systemd\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-tuned\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-registration-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-socket-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897475    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbfm4\" (UniqueName: \"kubernetes.io/projected/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-kube-api-access-wbfm4\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-run-netns\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-socket-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-log-socket\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/117de80d-dc8d-43d8-b406-85c3a636ecb4-serviceca\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-host\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbh2\" (UniqueName: \"kubernetes.io/projected/f3f6ffad-aaed-444f-89f9-63284a8f55de-kube-api-access-5fbh2\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-netns\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-device-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-etc-selinux\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-var-lib-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-log-socket\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-kubernetes\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f2089a3-9817-4c29-980b-f4018a7b145e-dbus\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3b656de9-f949-4ae4-b1a7-9ee77032b734-hosts-file\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-device-dir\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-kubelet\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-run-netns\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897881    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 18:10:33.898912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c74h\" (UniqueName: \"kubernetes.io/projected/05b6c004-f567-4008-9544-1fc839d96155-kube-api-access-7c74h\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.897993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-iptables-alerter-script\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-run-ovn-kubernetes\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c3e22d68-3089-47b8-b9b1-1c81db256a73-konnectivity-ca\") pod \"konnectivity-agent-4vhkv\" (UID: \"c3e22d68-3089-47b8-b9b1-1c81db256a73\") " pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/05b6c004-f567-4008-9544-1fc839d96155-etc-selinux\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/117de80d-dc8d-43d8-b406-85c3a636ecb4-serviceca\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8vg4\" (UniqueName: \"kubernetes.io/projected/117de80d-dc8d-43d8-b406-85c3a636ecb4-kube-api-access-m8vg4\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-k8s-cni-cncf-io\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-var-lib-openvswitch\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-hostroot\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898392    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovnkube-config\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c3e22d68-3089-47b8-b9b1-1c81db256a73-konnectivity-ca\") pod \"konnectivity-agent-4vhkv\" (UID: \"c3e22d68-3089-47b8-b9b1-1c81db256a73\") " pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-cni-netd\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898787    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-run-ovn-kubernetes\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovnkube-script-lib\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-host-cni-netd\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.899637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.898896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b656de9-f949-4ae4-b1a7-9ee77032b734-tmp-dir\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899014    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysctl-d\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-lib-modules\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-modprobe-d\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ffad-aaed-444f-89f9-63284a8f55de-tmp\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-multus-certs\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln456\" (UniqueName: \"kubernetes.io/projected/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-kube-api-access-ln456\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysctl-conf\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovnkube-script-lib\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b656de9-f949-4ae4-b1a7-9ee77032b734-tmp-dir\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-iptables-alerter-script\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.900476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.899735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.902896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.902743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c3e22d68-3089-47b8-b9b1-1c81db256a73-agent-certs\") pod \"konnectivity-agent-4vhkv\" (UID: \"c3e22d68-3089-47b8-b9b1-1c81db256a73\") " pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:33.903501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.903447    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-ovn-node-metrics-cert\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.912273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.912215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal" event={"ID":"9add31ed36a5b5cac8ed65d2c77f17bf","Type":"ContainerStarted","Data":"439715a12f4bcf75642a289bb2500b60464d1f0f53902ad5709c0ec1bc1ea4c1"}
Sep 11 18:10:33.914122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.913942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lllv\" (UniqueName: \"kubernetes.io/projected/a11915f1-cc9f-46ab-b2de-44c8aa7a2e86-kube-api-access-4lllv\") pod \"ovnkube-node-7bpkd\" (UID: \"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:33.914227 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.914186    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:10:33.914227 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.914210    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:10:33.914227 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.914224    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:33.914403 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:33.914305    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:34.414287313 +0000 UTC m=+3.007487871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:33.914668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.914627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" event={"ID":"3ff41c913f1d6f6987c1f137b7ff6119","Type":"ContainerStarted","Data":"99a7def03f44a4670fd5100d8ef564c1b1221de919fee6a0d6b578022feac8f1"}
Sep 11 18:10:33.915910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.915888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrc2v\" (UniqueName: \"kubernetes.io/projected/df55e017-4726-49f9-8507-41222acd67f0-kube-api-access-zrc2v\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:33.916723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.916492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbfm4\" (UniqueName: \"kubernetes.io/projected/9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e-kube-api-access-wbfm4\") pod \"iptables-alerter-szmgh\" (UID: \"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e\") " pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:33.917289 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.917249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8vg4\" (UniqueName: \"kubernetes.io/projected/117de80d-dc8d-43d8-b406-85c3a636ecb4-kube-api-access-m8vg4\") pod \"node-ca-4ft7z\" (UID: \"117de80d-dc8d-43d8-b406-85c3a636ecb4\") " pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:33.917935 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.917908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2x28\" (UniqueName: \"kubernetes.io/projected/3b656de9-f949-4ae4-b1a7-9ee77032b734-kube-api-access-t2x28\") pod \"node-resolver-x46v2\" (UID: \"3b656de9-f949-4ae4-b1a7-9ee77032b734\") " pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:33.918061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.918041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln456\" (UniqueName: \"kubernetes.io/projected/a3a338ce-174e-464b-8a7a-7fee4c5dd1ee-kube-api-access-ln456\") pod \"multus-additional-cni-plugins-4rdld\" (UID: \"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee\") " pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:33.918932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:33.918900    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c74h\" (UniqueName: \"kubernetes.io/projected/05b6c004-f567-4008-9544-1fc839d96155-kube-api-access-7c74h\") pod \"aws-ebs-csi-driver-node-bd5zx\" (UID: \"05b6c004-f567-4008-9544-1fc839d96155\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:34.000460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-kubernetes\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.000460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f2089a3-9817-4c29-980b-f4018a7b145e-dbus\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-kubelet\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-k8s-cni-cncf-io\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-hostroot\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000604    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-kubelet\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000616    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysctl-d\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f2089a3-9817-4c29-980b-f4018a7b145e-dbus\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-kubernetes\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-lib-modules\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-hostroot\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.000686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-k8s-cni-cncf-io\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-lib-modules\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-modprobe-d\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysctl-d\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ffad-aaed-444f-89f9-63284a8f55de-tmp\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-multus-certs\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-modprobe-d\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysctl-conf\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xxwv\" (UniqueName: \"kubernetes.io/projected/06168f80-3164-4c7e-a316-f7449b680db9-kube-api-access-9xxwv\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000835    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-run\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-cni-bin\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000819    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-multus-certs\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-etc-kubernetes\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06168f80-3164-4c7e-a316-f7449b680db9-multus-daemon-config\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-etc-kubernetes\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-sys\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-run\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-cni-bin\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.001191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-cni-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysctl-conf\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.000999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-cni-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-sys\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06168f80-3164-4c7e-a316-f7449b680db9-cni-binary-copy\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-socket-dir-parent\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysconfig\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-socket-dir-parent\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f2089a3-9817-4c29-980b-f4018a7b145e-kubelet-config\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-sysconfig\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-var-lib-kubelet\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f2089a3-9817-4c29-980b-f4018a7b145e-kubelet-config\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-cni-multus\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.001299    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-cnibin\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001314    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-var-lib-kubelet\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-system-cni-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-conf-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-var-lib-cni-multus\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.001382    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:10:34.501360848 +0000 UTC m=+3.094561403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-multus-conf-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-cnibin\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-systemd\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-system-cni-dir\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-tuned\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-systemd\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-host\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001523    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbh2\" (UniqueName: \"kubernetes.io/projected/f3f6ffad-aaed-444f-89f9-63284a8f55de-kube-api-access-5fbh2\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-netns\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06168f80-3164-4c7e-a316-f7449b680db9-cni-binary-copy\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/06168f80-3164-4c7e-a316-f7449b680db9-multus-daemon-config\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001580    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f3f6ffad-aaed-444f-89f9-63284a8f55de-host\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.002947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.001612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/06168f80-3164-4c7e-a316-f7449b680db9-host-run-netns\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.003637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.003527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ffad-aaed-444f-89f9-63284a8f55de-tmp\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.003637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.003599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f3f6ffad-aaed-444f-89f9-63284a8f55de-etc-tuned\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.012338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.012298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xxwv\" (UniqueName: \"kubernetes.io/projected/06168f80-3164-4c7e-a316-f7449b680db9-kube-api-access-9xxwv\") pod \"multus-9b54z\" (UID: \"06168f80-3164-4c7e-a316-f7449b680db9\") " pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.012782 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.012766    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbh2\" (UniqueName: \"kubernetes.io/projected/f3f6ffad-aaed-444f-89f9-63284a8f55de-kube-api-access-5fbh2\") pod \"tuned-wzld5\" (UID: \"f3f6ffad-aaed-444f-89f9-63284a8f55de\") " pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.086698 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.086645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x46v2"
Sep 11 18:10:34.095303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.095283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4ft7z"
Sep 11 18:10:34.105417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.105401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:34.112351 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.112332    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4rdld"
Sep 11 18:10:34.119826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.119810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx"
Sep 11 18:10:34.128325 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.128307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-szmgh"
Sep 11 18:10:34.135954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.135938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:34.141900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.141878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wzld5"
Sep 11 18:10:34.147435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.147419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9b54z"
Sep 11 18:10:34.404135 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.404068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:34.404287 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.404184    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:34.404287 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.404237    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:35.404221167 +0000 UTC m=+3.997421702 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:34.504969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.504938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:34.505134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.505001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.505134 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.505109    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:34.505247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.505170    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:10:35.505152994 +0000 UTC m=+4.098353542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:34.505247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.505111    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:10:34.505247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.505210    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:10:34.505247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.505222    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:34.505463 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.505286    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:35.505270273 +0000 UTC m=+4.098470823 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:34.856818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.856746    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 18:05:33 +0000 UTC" deadline="2028-02-25 16:06:04.275170044 +0000 UTC"
Sep 11 18:10:34.856818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.856775    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12765h55m29.418397883s"
Sep 11 18:10:34.906524 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.906491    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:34.906633 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:34.906614    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:34.910313 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:34.910156    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa7f7d6_5fd4_41ad_ad8b_b07da6acef2e.slice/crio-884f30dc4ec1ca8ff7942c31490cf163b34e198c964747585d7b5b7b27a25c1e WatchSource:0}: Error finding container 884f30dc4ec1ca8ff7942c31490cf163b34e198c964747585d7b5b7b27a25c1e: Status 404 returned error can't find the container with id 884f30dc4ec1ca8ff7942c31490cf163b34e198c964747585d7b5b7b27a25c1e
Sep 11 18:10:34.910732 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:34.910702    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11915f1_cc9f_46ab_b2de_44c8aa7a2e86.slice/crio-d4a3fc3038c9efabb7c662c14d0013cbf8725d67b3e37382a338343f50c55e84 WatchSource:0}: Error finding container d4a3fc3038c9efabb7c662c14d0013cbf8725d67b3e37382a338343f50c55e84: Status 404 returned error can't find the container with id d4a3fc3038c9efabb7c662c14d0013cbf8725d67b3e37382a338343f50c55e84
Sep 11 18:10:34.912732 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:34.912706    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e22d68_3089_47b8_b9b1_1c81db256a73.slice/crio-96026cf1e87199968b381b371b057e5398dda8c09828c92ed36f3019149d707e WatchSource:0}: Error finding container 96026cf1e87199968b381b371b057e5398dda8c09828c92ed36f3019149d707e: Status 404 returned error can't find the container with id 96026cf1e87199968b381b371b057e5398dda8c09828c92ed36f3019149d707e
Sep 11 18:10:34.915511 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:34.915490    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a338ce_174e_464b_8a7a_7fee4c5dd1ee.slice/crio-2e6b19edf5bcc2968a8dfaea58c8432c402cb2b5678ab3d3f34f704d413e3f79 WatchSource:0}: Error finding container 2e6b19edf5bcc2968a8dfaea58c8432c402cb2b5678ab3d3f34f704d413e3f79: Status 404 returned error can't find the container with id 2e6b19edf5bcc2968a8dfaea58c8432c402cb2b5678ab3d3f34f704d413e3f79
Sep 11 18:10:34.918497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.918470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"d4a3fc3038c9efabb7c662c14d0013cbf8725d67b3e37382a338343f50c55e84"}
Sep 11 18:10:34.923712 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:10:34.922541    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117de80d_dc8d_43d8_b406_85c3a636ecb4.slice/crio-97b7689b207be51c34da271b3946e8f57ba2e751fda49a2d2c7f0cea1dcae141 WatchSource:0}: Error finding container 97b7689b207be51c34da271b3946e8f57ba2e751fda49a2d2c7f0cea1dcae141: Status 404 returned error can't find the container with id 97b7689b207be51c34da271b3946e8f57ba2e751fda49a2d2c7f0cea1dcae141
Sep 11 18:10:34.926594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.926448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-szmgh" event={"ID":"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e","Type":"ContainerStarted","Data":"884f30dc4ec1ca8ff7942c31490cf163b34e198c964747585d7b5b7b27a25c1e"}
Sep 11 18:10:34.931976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:34.931943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4vhkv" event={"ID":"c3e22d68-3089-47b8-b9b1-1c81db256a73","Type":"ContainerStarted","Data":"96026cf1e87199968b381b371b057e5398dda8c09828c92ed36f3019149d707e"}
Sep 11 18:10:35.412678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.412361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:35.412678 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.412555    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:35.412678 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.412624    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:37.412605283 +0000 UTC m=+6.005805832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:35.513520 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.513484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:35.513668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.513564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:35.513741 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.513695    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:35.513800 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.513754    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:10:37.513735091 +0000 UTC m=+6.106935640 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:35.514159 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.514138    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:10:35.514247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.514163    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:10:35.514247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.514176    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:35.514247 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.514222    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:37.514206225 +0000 UTC m=+6.107406773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:35.857413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.857310    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 18:05:33 +0000 UTC" deadline="2028-04-19 11:59:47.563647872 +0000 UTC"
Sep 11 18:10:35.857413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.857368    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14057h49m11.706284s"
Sep 11 18:10:35.907758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.907278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:35.907758 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.907415    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:35.908166 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.908038    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:35.908166 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:35.908132    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:35.942051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.941989    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x46v2" event={"ID":"3b656de9-f949-4ae4-b1a7-9ee77032b734","Type":"ContainerStarted","Data":"35b2697d04747a199d7950ee59274c6c7e5b1e795c5296d86e526d276e164689"}
Sep 11 18:10:35.959279 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.959175    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx" event={"ID":"05b6c004-f567-4008-9544-1fc839d96155","Type":"ContainerStarted","Data":"b9a2949e002be962e5ce2b26ea493b8bcf4d5de54d16d98bf38ab6e3f48cd554"}
Sep 11 18:10:35.983363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.982665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal" event={"ID":"9add31ed36a5b5cac8ed65d2c77f17bf","Type":"ContainerStarted","Data":"9e475516bcbffe73473e8ecc91eea858aa78e2b58745b7c49f91fdd01d1e2761"}
Sep 11 18:10:35.996680 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.996650    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ff41c913f1d6f6987c1f137b7ff6119" containerID="cae1e62067456832f2fe16d4484d0014a462bbb15dd7c2b5307de301b237700f" exitCode=0
Sep 11 18:10:35.996859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:35.996733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" event={"ID":"3ff41c913f1d6f6987c1f137b7ff6119","Type":"ContainerDied","Data":"cae1e62067456832f2fe16d4484d0014a462bbb15dd7c2b5307de301b237700f"}
Sep 11 18:10:36.005338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:36.005287    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-142.ec2.internal" podStartSLOduration=3.005269882 podStartE2EDuration="3.005269882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:36.00405778 +0000 UTC m=+4.597258330" watchObservedRunningTime="2026-09-11 18:10:36.005269882 +0000 UTC m=+4.598470445"
Sep 11 18:10:36.014740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:36.014689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wzld5" event={"ID":"f3f6ffad-aaed-444f-89f9-63284a8f55de","Type":"ContainerStarted","Data":"e1480efaa231174f57dba7ecaf346fe2d5ad15e22fe0a7733a29f2b2a4900fbc"}
Sep 11 18:10:36.023282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:36.023247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ft7z" event={"ID":"117de80d-dc8d-43d8-b406-85c3a636ecb4","Type":"ContainerStarted","Data":"97b7689b207be51c34da271b3946e8f57ba2e751fda49a2d2c7f0cea1dcae141"}
Sep 11 18:10:36.049166 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:36.049133    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9b54z" event={"ID":"06168f80-3164-4c7e-a316-f7449b680db9","Type":"ContainerStarted","Data":"ab9425925af7d894c3e87a970290b704208173c5fb019f109a32ef44e20440e0"}
Sep 11 18:10:36.070236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:36.070167    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerStarted","Data":"2e6b19edf5bcc2968a8dfaea58c8432c402cb2b5678ab3d3f34f704d413e3f79"}
Sep 11 18:10:36.906944 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:36.906914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:36.907549 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:36.907039    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:37.093097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.092949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" event={"ID":"3ff41c913f1d6f6987c1f137b7ff6119","Type":"ContainerStarted","Data":"ff006541ba4db50e1b036d8e10cb5742c3c81c98d8c6de46ed11412ad27fc04a"}
Sep 11 18:10:37.117687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.117375    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-142.ec2.internal" podStartSLOduration=4.117354793 podStartE2EDuration="4.117354793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:37.115356916 +0000 UTC m=+5.708557486" watchObservedRunningTime="2026-09-11 18:10:37.117354793 +0000 UTC m=+5.710555351"
Sep 11 18:10:37.431027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.430948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:37.431195 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.431140    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:37.431249 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.431201    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:41.431182835 +0000 UTC m=+10.024383386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:37.531551 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.531490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:37.531713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.531602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:37.531776 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.531734    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:37.531832 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.531799    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:10:41.531781819 +0000 UTC m=+10.124982372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:37.531988 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.531886    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:10:37.531988 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.531906    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:10:37.531988 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.531919    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:37.531988 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.531964    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:41.531951214 +0000 UTC m=+10.125151764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:37.909189 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.909118    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:37.909624 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.909252    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:37.909624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:37.909613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:37.909737 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:37.909695    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:38.906295 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:38.906248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:38.906478 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:38.906387    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:39.906364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:39.906307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:39.906962 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:39.906433    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:39.906962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:39.906246    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:39.906962 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:39.906855    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:40.907481 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:40.906762    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:40.907481 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:40.907095    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:41.463061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:41.463008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:41.463249 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.463224    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:41.463366 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.463305    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:49.463284838 +0000 UTC m=+18.056485385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:41.564208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:41.564163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:41.564365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:41.564234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:41.564415 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.564389    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:10:41.564415 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.564410    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:10:41.564509 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.564423    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:41.564509 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.564490    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:10:49.564462076 +0000 UTC m=+18.157662629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:41.564849 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.564742    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:41.564849 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.564821    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:10:49.564803399 +0000 UTC m=+18.158003950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:41.907556 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:41.907458    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:41.907990 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.907679    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:41.907990 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:41.907696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:41.907990 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:41.907785    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:42.906440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:42.906409    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:42.906581 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:42.906541    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:43.906186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:43.906075    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:43.906660 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:43.906220    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:43.906660 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:43.906287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:43.906660 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:43.906391    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:44.906829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:44.906800    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:44.907286 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:44.906915    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:45.906284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:45.906245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:45.906465 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:45.906400    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:45.906465 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:45.906422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:45.906574 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:45.906529    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:46.906857 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:46.906824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:46.907277 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:46.906934    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:47.906250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:47.906193    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:47.906426 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:47.906332    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:47.906426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:47.906393    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:47.906534 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:47.906511    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:48.906629 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:48.906557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:48.907112 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:48.906680    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:49.523232 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:49.523203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:49.523404 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.523378    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:49.523459 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.523453    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:05.523431968 +0000 UTC m=+34.116632510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:10:49.624320 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:49.624288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:49.624488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:49.624353    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:49.624488 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.624452    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:49.624592 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.624481    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:10:49.624592 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.624507    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:11:05.624490449 +0000 UTC m=+34.217691009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:10:49.624592 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.624507    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:10:49.624592 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.624524    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:49.624592 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.624580    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:05.624563893 +0000 UTC m=+34.217764430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:10:49.906677 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:49.906606    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:49.907100 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.906729    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:49.907100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:49.906782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:49.907100 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:49.906883    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:50.906790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:50.906757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:50.907198 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:50.906865    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:51.909600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:51.909573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:51.909600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:51.909586    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:51.910032 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:51.909670    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:51.910032 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:51.909803    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:52.907063 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:52.906932    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:52.907194 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:52.907170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:53.122529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.122502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx" event={"ID":"05b6c004-f567-4008-9544-1fc839d96155","Type":"ContainerStarted","Data":"20b3bcc8f75c61ec0cd2580dbdd9c0b2676b77a4ea6292ee259b83188c11f760"}
Sep 11 18:10:53.123798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.123764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4vhkv" event={"ID":"c3e22d68-3089-47b8-b9b1-1c81db256a73","Type":"ContainerStarted","Data":"2925ca26f96423935a100e5224ffb411d1f441c92f45e76334a520fc12eca06f"}
Sep 11 18:10:53.125320 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.125290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wzld5" event={"ID":"f3f6ffad-aaed-444f-89f9-63284a8f55de","Type":"ContainerStarted","Data":"6bd9c57ba8dd2a048f517086ea088931c9c1dcfb3e852e31a2581b3a760b9613"}
Sep 11 18:10:53.126808 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.126786    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4ft7z" event={"ID":"117de80d-dc8d-43d8-b406-85c3a636ecb4","Type":"ContainerStarted","Data":"dc997ac31084225b463f19cffedc2f3c33f43e9dbae7420a2ef9d9b7dc1ff2f7"}
Sep 11 18:10:53.128398 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.128375    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9b54z" event={"ID":"06168f80-3164-4c7e-a316-f7449b680db9","Type":"ContainerStarted","Data":"f6810e9147e6c0c444a64479009ae3cc612bf9891cec5291cd8d5cd79ec56535"}
Sep 11 18:10:53.130208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.130185    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3a338ce-174e-464b-8a7a-7fee4c5dd1ee" containerID="911f193344d2880b71276d66f379e26cb1365ada27284ecd088a8fea981949d5" exitCode=0
Sep 11 18:10:53.130299 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.130281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerDied","Data":"911f193344d2880b71276d66f379e26cb1365ada27284ecd088a8fea981949d5"}
Sep 11 18:10:53.133529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.133513    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:10:53.133931 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.133907    2563 generic.go:356] "Generic (PLEG): container finished" podID="a11915f1-cc9f-46ab-b2de-44c8aa7a2e86" containerID="e79747b70727f25ffadd8b577f1438572a1366b79bf6c8aaecc8c082bb156847" exitCode=1
Sep 11 18:10:53.134068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.133978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"2237acce719c725ffcaa57300b895f5e3437825e6a3dfc85f8dfb3e6d8c367f6"}
Sep 11 18:10:53.134068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.134059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"9088a0f714a1cead895d378225970842ecc2c43d890867cc8641ad9c513abb35"}
Sep 11 18:10:53.134163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.134110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"864de0d6c7d621877e4aa7f87cd276f62d3d176a2a057db1d06458fa929793b8"}
Sep 11 18:10:53.134163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.134159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"b9b4902001a911152ab6c2e6f4c0eab98d54bda1c7c2a72e0cfaf8d689dbeb56"}
Sep 11 18:10:53.134245 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.134212    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerDied","Data":"e79747b70727f25ffadd8b577f1438572a1366b79bf6c8aaecc8c082bb156847"}
Sep 11 18:10:53.134313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.134281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"bb223b3dbc8cb9939071e67ef89512ca91ac9bb6988f7114250550a5bd1f9930"}
Sep 11 18:10:53.135551 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.135531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x46v2" event={"ID":"3b656de9-f949-4ae4-b1a7-9ee77032b734","Type":"ContainerStarted","Data":"ea21429c3a9c79e6a9d500df2a3d988b35208cb19b5327c82f0c71c9f7aa19f5"}
Sep 11 18:10:53.149237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.149202    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4vhkv" podStartSLOduration=4.50523365 podStartE2EDuration="22.149189757s" totalImagesPullingTime="17.643956107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:53.149089641 +0000 UTC m=+21.742290198" watchObservedRunningTime="2026-09-11 18:10:53.149189757 +0000 UTC m=+21.742390343"
Sep 11 18:10:53.171809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.171777    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4ft7z" podStartSLOduration=4.554678812 podStartE2EDuration="22.171767296s" totalImagesPullingTime="17.617088484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:53.171738488 +0000 UTC m=+21.764939044" watchObservedRunningTime="2026-09-11 18:10:53.171767296 +0000 UTC m=+21.764967849"
Sep 11 18:10:53.230504 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.230438    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x46v2" podStartSLOduration=8.626727332 podStartE2EDuration="22.230427164s" totalImagesPullingTime="13.603699832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:53.230376403 +0000 UTC m=+21.823576971" watchObservedRunningTime="2026-09-11 18:10:53.230427164 +0000 UTC m=+21.823627720"
Sep 11 18:10:53.315418 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.315379    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wzld5" podStartSLOduration=3.686072624 podStartE2EDuration="21.315367666s" totalImagesPullingTime="17.629295042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:53.274635728 +0000 UTC m=+21.867836284" watchObservedRunningTime="2026-09-11 18:10:53.315367666 +0000 UTC m=+21.908568223"
Sep 11 18:10:53.315526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.315508    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9b54z" podStartSLOduration=3.464355404 podStartE2EDuration="21.315502546s" totalImagesPullingTime="17.851147142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:53.3153503 +0000 UTC m=+21.908550866" watchObservedRunningTime="2026-09-11 18:10:53.315502546 +0000 UTC m=+21.908703104"
Sep 11 18:10:53.347587 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.347563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:53.487611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.487544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:53.569183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.569159    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:53.910186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.910163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:53.910439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:53.910197    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:53.910439 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:53.910283    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:53.910439 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:53.910353    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:54.139976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.139939    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-szmgh" event={"ID":"9fa7f7d6-5fd4-41ad-ad8b-b07da6acef2e","Type":"ContainerStarted","Data":"c955938b6d98f7e70e7637842bdc0dc28906c3793564fc9283def09818d55199"}
Sep 11 18:10:54.159628 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.159555    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-szmgh" podStartSLOduration=8.547987306 podStartE2EDuration="22.159530011s" totalImagesPullingTime="13.611542705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:54.158667822 +0000 UTC m=+22.751868382" watchObservedRunningTime="2026-09-11 18:10:54.159530011 +0000 UTC m=+22.752730560"
Sep 11 18:10:54.284535 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.284511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:54.381929 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.381896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:54.507611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.507527    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 18:10:54.846310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.846137    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T18:10:54.507543874Z","UUID":"3d987f3b-0fbf-4388-a17b-db1a2850f445","Handler":null,"Name":"","Endpoint":""}
Sep 11 18:10:54.848115 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.848089    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 18:10:54.848251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.848139    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 18:10:54.906407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:54.906380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:54.906548 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:54.906502    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:55.144449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:55.144364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx" event={"ID":"05b6c004-f567-4008-9544-1fc839d96155","Type":"ContainerStarted","Data":"acbb63dd1c5fcab168b5f74aa8e778ce2f31ebfd87ce7511f677b8de4afc578f"}
Sep 11 18:10:55.906776 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:55.906714    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:55.906776 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:55.906776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:55.907025 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:55.906879    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:55.907084 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:55.907024    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:56.037949 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.037897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:56.134243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.134210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:56.150489 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.150463    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:10:56.150940 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.150916    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"51519c86d18c096e80452881220de5b43e03ff2c014cabb245e121f5cd525518"}
Sep 11 18:10:56.153372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.153346    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx" event={"ID":"05b6c004-f567-4008-9544-1fc839d96155","Type":"ContainerStarted","Data":"556780acb5cc520a3637051c649aa9c6220f838ac3538143f268eac043cc8389"}
Sep 11 18:10:56.250569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.250512    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:56.251354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.251334    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:56.273254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.273208    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bd5zx" podStartSLOduration=4.409148479 podStartE2EDuration="24.273189888s" totalImagesPullingTime="19.864041409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:56.175107879 +0000 UTC m=+24.768308436" watchObservedRunningTime="2026-09-11 18:10:56.273189888 +0000 UTC m=+24.866390447"
Sep 11 18:10:56.906208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:56.906178    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:56.906365 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:56.906289    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:57.156473 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:57.156394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:57.156962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:57.156782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4vhkv"
Sep 11 18:10:57.230364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:57.230333    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:57.906803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:57.906774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:57.906971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:57.906916    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:57.907086 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:57.907051    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:57.907086 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:57.906910    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:58.161911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.161682    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:10:58.162476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.162313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"993eeb88f31a500646b87a78ae238d4706f86b8a5fa7bb262056e1b66f6aedc2"}
Sep 11 18:10:58.162670 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.162651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:58.162769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.162720    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:58.162769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.162743    2563 scope.go:117] "RemoveContainer" containerID="e79747b70727f25ffadd8b577f1438572a1366b79bf6c8aaecc8c082bb156847"
Sep 11 18:10:58.183528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.183502    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:58.906139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:58.906107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:58.906316 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:58.906203    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:10:59.167497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.167448    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:10:59.168004 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.167813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" event={"ID":"a11915f1-cc9f-46ab-b2de-44c8aa7a2e86","Type":"ContainerStarted","Data":"8789fe1aaf0e9f7af1e3368315fccd800b3c10db7eccf4d15a6c0de4a99ef867"}
Sep 11 18:10:59.168106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.168083    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:59.169707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.169680    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3a338ce-174e-464b-8a7a-7fee4c5dd1ee" containerID="bf17ec12ebeedbee3a60781e9bf9a32f66f12026fae55bf71b8d40ebcbf248d9" exitCode=0
Sep 11 18:10:59.169818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.169767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerDied","Data":"bf17ec12ebeedbee3a60781e9bf9a32f66f12026fae55bf71b8d40ebcbf248d9"}
Sep 11 18:10:59.183041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.183022    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:10:59.202458 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.202421    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd" podStartSLOduration=9.53957237 podStartE2EDuration="27.202408624s" totalImagesPullingTime="17.662836254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:59.201495109 +0000 UTC m=+27.794695679" watchObservedRunningTime="2026-09-11 18:10:59.202408624 +0000 UTC m=+27.795609164"
Sep 11 18:10:59.906567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.906402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:10:59.906732 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.906410    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:10:59.906732 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:59.906672    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:10:59.906732 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:59.906708    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:10:59.940774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.940733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qhmz2"]
Sep 11 18:10:59.943656 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.943633    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m67bj"]
Sep 11 18:10:59.944370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.944347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6ntmn"]
Sep 11 18:10:59.944470 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:10:59.944429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:10:59.944550 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:10:59.944522    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:11:00.174273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:00.174232    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3a338ce-174e-464b-8a7a-7fee4c5dd1ee" containerID="6319d583341d1f8fb656d3d69099f487b0c9326976be6888f7ac680745b0dece" exitCode=0
Sep 11 18:11:00.174617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:00.174290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerDied","Data":"6319d583341d1f8fb656d3d69099f487b0c9326976be6888f7ac680745b0dece"}
Sep 11 18:11:00.174617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:00.174373    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:00.174617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:00.174380    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:11:00.174617 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:00.174559    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:11:00.174766 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:00.174624    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:11:00.365163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:00.365138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:00.429179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:00.429155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:01.179324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.179226    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3a338ce-174e-464b-8a7a-7fee4c5dd1ee" containerID="150720def8f7343eefdb305ad17b245358809502305cbc139e4a167ce2e7b135" exitCode=0
Sep 11 18:11:01.179717 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.179316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerDied","Data":"150720def8f7343eefdb305ad17b245358809502305cbc139e4a167ce2e7b135"}
Sep 11 18:11:01.208109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.208075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:01.353651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.353620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:01.907311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.907287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:11:01.907461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.907368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:01.907461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:01.907398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:01.907551 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:01.907474    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:11:01.907551 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:01.907368    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:11:01.907668 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:01.907552    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:11:03.268013 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:03.267972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:03.369315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:03.369280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:03.906656 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:03.906623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:03.906828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:03.906623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:11:03.906828 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:03.906732    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m67bj" podUID="f65abfcd-9d97-4224-958d-f4aeb2d78668"
Sep 11 18:11:03.906828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:03.906623    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:03.906986 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:03.906825    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhmz2" podUID="df55e017-4726-49f9-8507-41222acd67f0"
Sep 11 18:11:03.907053 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:03.907027    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ntmn" podUID="5f2089a3-9817-4c29-980b-f4018a7b145e"
Sep 11 18:11:05.547527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.547491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:11:05.547966 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.547679    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:11:05.547966 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.547754    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:37.547737941 +0000 UTC m=+66.140938489 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 18:11:05.648324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.648295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:05.648446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.648342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:05.648446 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.648431    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 18:11:05.648575 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.648445    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 18:11:05.648575 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.648458    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 18:11:05.648575 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.648467    2563 projected.go:196] Error preparing data for projected volume kube-api-access-fwdck for pod openshift-network-diagnostics/network-check-target-m67bj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:11:05.648575 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.648486    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret podName:5f2089a3-9817-4c29-980b-f4018a7b145e nodeName:}" failed. No retries permitted until 2026-09-11 18:11:37.648472737 +0000 UTC m=+66.241673284 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret") pod "global-pull-secret-syncer-6ntmn" (UID: "5f2089a3-9817-4c29-980b-f4018a7b145e") : object "kube-system"/"original-pull-secret" not registered
Sep 11 18:11:05.648575 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.648507    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck podName:f65abfcd-9d97-4224-958d-f4aeb2d78668 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:37.648496589 +0000 UTC m=+66.241697124 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fwdck" (UniqueName: "kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck") pod "network-check-target-m67bj" (UID: "f65abfcd-9d97-4224-958d-f4aeb2d78668") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 18:11:05.666180 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.666160    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-142.ec2.internal" event="NodeReady"
Sep 11 18:11:05.666328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.666248    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 18:11:05.723769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.723742    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vk549"]
Sep 11 18:11:05.737611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.737591    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rjkqn"]
Sep 11 18:11:05.737717 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.737701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.740890 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.740867    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 18:11:05.740971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.740956    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4zn8w\""
Sep 11 18:11:05.747422 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.747406    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 18:11:05.751809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.751790    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vk549"]
Sep 11 18:11:05.751905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.751829    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rjkqn"]
Sep 11 18:11:05.751964 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.751912    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:05.754879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.754861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 18:11:05.754968 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.754905    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 18:11:05.755502 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.755481    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 18:11:05.755600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.755551    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-p7bc6\""
Sep 11 18:11:05.849027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.848977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.849027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.849013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50bc98-cf61-4994-bcfd-9c43af9229e4-config-volume\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.849160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.849038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0266d274-0277-49ea-ac36-9e8bf474fae3-cert\") pod \"ingress-canary-rjkqn\" (UID: \"0266d274-0277-49ea-ac36-9e8bf474fae3\") " pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:05.849160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.849095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b50bc98-cf61-4994-bcfd-9c43af9229e4-tmp-dir\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.849160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.849130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lcv5\" (UniqueName: \"kubernetes.io/projected/6b50bc98-cf61-4994-bcfd-9c43af9229e4-kube-api-access-8lcv5\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.849160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.849146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l6fk\" (UniqueName: \"kubernetes.io/projected/0266d274-0277-49ea-ac36-9e8bf474fae3-kube-api-access-8l6fk\") pod \"ingress-canary-rjkqn\" (UID: \"0266d274-0277-49ea-ac36-9e8bf474fae3\") " pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:05.909318 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.909299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:05.909403 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.909370    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:05.909524 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.909501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:11:05.911693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.911669    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztbbx\""
Sep 11 18:11:05.911787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.911715    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 18:11:05.911787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.911678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 18:11:05.911891 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.911810    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 18:11:05.912039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.912024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 18:11:05.914159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.914138    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bvnsl\""
Sep 11 18:11:05.948150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.948129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:05.950089 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0266d274-0277-49ea-ac36-9e8bf474fae3-cert\") pod \"ingress-canary-rjkqn\" (UID: \"0266d274-0277-49ea-ac36-9e8bf474fae3\") " pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:05.950147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b50bc98-cf61-4994-bcfd-9c43af9229e4-tmp-dir\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.950147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lcv5\" (UniqueName: \"kubernetes.io/projected/6b50bc98-cf61-4994-bcfd-9c43af9229e4-kube-api-access-8lcv5\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.950147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l6fk\" (UniqueName: \"kubernetes.io/projected/0266d274-0277-49ea-ac36-9e8bf474fae3-kube-api-access-8l6fk\") pod \"ingress-canary-rjkqn\" (UID: \"0266d274-0277-49ea-ac36-9e8bf474fae3\") " pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:05.950328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.950328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950225    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50bc98-cf61-4994-bcfd-9c43af9229e4-config-volume\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.950428 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.950360    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:05.950428 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:05.950419    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:06.450400817 +0000 UTC m=+35.043601370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:05.950623 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b50bc98-cf61-4994-bcfd-9c43af9229e4-tmp-dir\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.951008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.950838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b50bc98-cf61-4994-bcfd-9c43af9229e4-config-volume\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.953997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.953980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0266d274-0277-49ea-ac36-9e8bf474fae3-cert\") pod \"ingress-canary-rjkqn\" (UID: \"0266d274-0277-49ea-ac36-9e8bf474fae3\") " pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:05.958702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.958684    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lcv5\" (UniqueName: \"kubernetes.io/projected/6b50bc98-cf61-4994-bcfd-9c43af9229e4-kube-api-access-8lcv5\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:05.959367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:05.959346    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l6fk\" (UniqueName: \"kubernetes.io/projected/0266d274-0277-49ea-ac36-9e8bf474fae3-kube-api-access-8l6fk\") pod \"ingress-canary-rjkqn\" (UID: \"0266d274-0277-49ea-ac36-9e8bf474fae3\") " pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:06.044070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:06.044051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:06.061101 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:06.061079    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rjkqn"
Sep 11 18:11:06.195940 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:06.195915    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerStarted","Data":"f4ba54a95639d2fd2526d0419d48a70269163ccfe0f6c090b26fdf6e03c741fc"}
Sep 11 18:11:06.384009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:06.383929    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rjkqn"]
Sep 11 18:11:06.387844 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:06.387820    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0266d274_0277_49ea_ac36_9e8bf474fae3.slice/crio-bb8f16189ff03560eca9065d6ef46ac5cae11dff2071d213a6a170370a2e4187 WatchSource:0}: Error finding container bb8f16189ff03560eca9065d6ef46ac5cae11dff2071d213a6a170370a2e4187: Status 404 returned error can't find the container with id bb8f16189ff03560eca9065d6ef46ac5cae11dff2071d213a6a170370a2e4187
Sep 11 18:11:06.454657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:06.454634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:06.454763 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:06.454750    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:06.454805 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:06.454797    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:07.454782761 +0000 UTC m=+36.047983296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:07.201343 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:07.201309    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3a338ce-174e-464b-8a7a-7fee4c5dd1ee" containerID="f4ba54a95639d2fd2526d0419d48a70269163ccfe0f6c090b26fdf6e03c741fc" exitCode=0
Sep 11 18:11:07.202183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:07.201399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerDied","Data":"f4ba54a95639d2fd2526d0419d48a70269163ccfe0f6c090b26fdf6e03c741fc"}
Sep 11 18:11:07.202937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:07.202909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rjkqn" event={"ID":"0266d274-0277-49ea-ac36-9e8bf474fae3","Type":"ContainerStarted","Data":"bb8f16189ff03560eca9065d6ef46ac5cae11dff2071d213a6a170370a2e4187"}
Sep 11 18:11:07.464696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:07.464500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:07.464818 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:07.464611    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:07.464818 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:07.464776    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:09.464757471 +0000 UTC m=+38.057958012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:08.208896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:08.208860    2563 generic.go:356] "Generic (PLEG): container finished" podID="a3a338ce-174e-464b-8a7a-7fee4c5dd1ee" containerID="b4caf9149d1b489787193858fa7640e5cb52fde52d04df49928f9ec623dac48d" exitCode=0
Sep 11 18:11:08.209297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:08.208924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerDied","Data":"b4caf9149d1b489787193858fa7640e5cb52fde52d04df49928f9ec623dac48d"}
Sep 11 18:11:09.215243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:09.215214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rdld" event={"ID":"a3a338ce-174e-464b-8a7a-7fee4c5dd1ee","Type":"ContainerStarted","Data":"af5171eac7c77610a1bbf8d13ff9f42aefd2f8dd2ab74341a514c8d0aeba9d2a"}
Sep 11 18:11:09.238467 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:09.238421    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4rdld" podStartSLOduration=9.236318483 podStartE2EDuration="38.238400475s" totalImagesPullingTime="29.002081992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:09.237953666 +0000 UTC m=+37.831154224" watchObservedRunningTime="2026-09-11 18:11:09.238400475 +0000 UTC m=+37.831601033"
Sep 11 18:11:09.363838 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:09.363803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:09.431860 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:09.431832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:09.480922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:09.480872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:09.481016 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:09.480972    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:09.481053 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:09.481028    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:13.481015984 +0000 UTC m=+42.074216520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:10.219867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:10.219829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rjkqn" event={"ID":"0266d274-0277-49ea-ac36-9e8bf474fae3","Type":"ContainerStarted","Data":"9360406aa55351d01f1c5c80c965a99a4f0fb5e197cbcd757421d981afe8e776"}
Sep 11 18:11:10.239827 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:10.239783    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rjkqn" podStartSLOduration=2.523105413 podStartE2EDuration="5.239768466s" totalImagesPullingTime="2.716663053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:10.239129583 +0000 UTC m=+38.832330144" watchObservedRunningTime="2026-09-11 18:11:10.239768466 +0000 UTC m=+38.832969023"
Sep 11 18:11:10.863060 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:10.863027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:10.947848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:10.947824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:13.507231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:13.507191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:13.507642 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:13.507359    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:13.507642 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:13.507442    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:21.507425979 +0000 UTC m=+50.100626514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:21.559004 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:21.558963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:21.559452 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:21.559117    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:21.559452 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:21.559181    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:37.559165505 +0000 UTC m=+66.152366041 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:23.360405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:23.360372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:23.538300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:23.538252    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:26.702147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.702105    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"]
Sep 11 18:11:26.704244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.704218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"
Sep 11 18:11:26.706454 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.706430    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-mqt2q\""
Sep 11 18:11:26.707199 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.707170    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:11:26.707361 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.707253    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 18:11:26.716437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.716410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"]
Sep 11 18:11:26.794398 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.794365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqshk\" (UniqueName: \"kubernetes.io/projected/72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2-kube-api-access-hqshk\") pod \"volume-data-source-validator-86695b7cc7-vtq6n\" (UID: \"72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"
Sep 11 18:11:26.817721 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.817692    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"]
Sep 11 18:11:26.819505 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.819487    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.821870 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.821836    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:11:26.821994 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.821941    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 18:11:26.822061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.821987    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 18:11:26.822061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.821997    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-g9ptl\""
Sep 11 18:11:26.822190 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.822062    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 18:11:26.824172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.824149    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-jkm2r"]
Sep 11 18:11:26.827302 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.827244    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.827570 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.827424    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"]
Sep 11 18:11:26.829424 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.829406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.829918 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.829897    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 18:11:26.830000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.829934    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-786q9\""
Sep 11 18:11:26.830051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.830005    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 18:11:26.834859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.834833    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 18:11:26.835839 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.835820    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 18:11:26.836599 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.836563    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 18:11:26.836714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.836641    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 18:11:26.836980 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.836953    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 18:11:26.837073 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.837020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dj9fm\""
Sep 11 18:11:26.837169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.837131    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 18:11:26.837945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.837912    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"]
Sep 11 18:11:26.841712 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.841686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 18:11:26.843741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.843713    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-jkm2r"]
Sep 11 18:11:26.848306 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.848284    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"]
Sep 11 18:11:26.894730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8035ce1c-4e18-4f71-bbd7-65643b813d0e-service-ca-bundle\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.894832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.894832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrs9q\" (UniqueName: \"kubernetes.io/projected/f6687a09-2cda-40ca-8023-71593b98bf54-kube-api-access-zrs9q\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.894932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8035ce1c-4e18-4f71-bbd7-65643b813d0e-snapshots\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.894932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8035ce1c-4e18-4f71-bbd7-65643b813d0e-serving-cert\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.894932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp6xb\" (UniqueName: \"kubernetes.io/projected/05b2274a-8be6-4170-9723-81ff3601f471-kube-api-access-tp6xb\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.894932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/05b2274a-8be6-4170-9723-81ff3601f471-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.895118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqshk\" (UniqueName: \"kubernetes.io/projected/72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2-kube-api-access-hqshk\") pod \"volume-data-source-validator-86695b7cc7-vtq6n\" (UID: \"72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"
Sep 11 18:11:26.895118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.894964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8035ce1c-4e18-4f71-bbd7-65643b813d0e-tmp\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.895118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.895005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8035ce1c-4e18-4f71-bbd7-65643b813d0e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.895118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.895063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6687a09-2cda-40ca-8023-71593b98bf54-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.895314 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.895093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnl8m\" (UniqueName: \"kubernetes.io/projected/8035ce1c-4e18-4f71-bbd7-65643b813d0e-kube-api-access-lnl8m\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.895314 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.895152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6687a09-2cda-40ca-8023-71593b98bf54-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.905328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.905301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqshk\" (UniqueName: \"kubernetes.io/projected/72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2-kube-api-access-hqshk\") pod \"volume-data-source-validator-86695b7cc7-vtq6n\" (UID: \"72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"
Sep 11 18:11:26.929315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.929291    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-4n827"]
Sep 11 18:11:26.931080 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.931067    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827"
Sep 11 18:11:26.933279 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.933248    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-kvmk9\""
Sep 11 18:11:26.941766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.941747    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"]
Sep 11 18:11:26.943371 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.943355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:26.945760 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.945720    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 18:11:26.945760 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.945750    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-sqn5h\""
Sep 11 18:11:26.946214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.946023    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 18:11:26.946329 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.946227    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 18:11:26.946585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.946567    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-4n827"]
Sep 11 18:11:26.947526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.947510    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:11:26.965686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.965637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"]
Sep 11 18:11:26.995883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.995862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnl8m\" (UniqueName: \"kubernetes.io/projected/8035ce1c-4e18-4f71-bbd7-65643b813d0e-kube-api-access-lnl8m\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.995971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.995891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6687a09-2cda-40ca-8023-71593b98bf54-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.995971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.995910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8035ce1c-4e18-4f71-bbd7-65643b813d0e-service-ca-bundle\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.995971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.995934    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.995971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.995961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7060fb14-474d-48f5-9e21-9067715c8951-config\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:26.996172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrs9q\" (UniqueName: \"kubernetes.io/projected/f6687a09-2cda-40ca-8023-71593b98bf54-kube-api-access-zrs9q\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.996172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7060fb14-474d-48f5-9e21-9067715c8951-serving-cert\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:26.996172 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:26.996098    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:26.996172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdncn\" (UniqueName: \"kubernetes.io/projected/7060fb14-474d-48f5-9e21-9067715c8951-kube-api-access-gdncn\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:26.996172 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:26.996162    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls podName:05b2274a-8be6-4170-9723-81ff3601f471 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:27.496141316 +0000 UTC m=+56.089341863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6hwgd" (UID: "05b2274a-8be6-4170-9723-81ff3601f471") : secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:26.996430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8035ce1c-4e18-4f71-bbd7-65643b813d0e-snapshots\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.996430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8035ce1c-4e18-4f71-bbd7-65643b813d0e-serving-cert\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.996430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp6xb\" (UniqueName: \"kubernetes.io/projected/05b2274a-8be6-4170-9723-81ff3601f471-kube-api-access-tp6xb\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.996430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/05b2274a-8be6-4170-9723-81ff3601f471-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.996430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccl56\" (UniqueName: \"kubernetes.io/projected/fb828f6e-099c-437c-9501-7f9e26941795-kube-api-access-ccl56\") pod \"network-check-source-6f8b768455-4n827\" (UID: \"fb828f6e-099c-437c-9501-7f9e26941795\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827"
Sep 11 18:11:26.996430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8035ce1c-4e18-4f71-bbd7-65643b813d0e-tmp\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.996701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8035ce1c-4e18-4f71-bbd7-65643b813d0e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.996701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6687a09-2cda-40ca-8023-71593b98bf54-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.996701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8035ce1c-4e18-4f71-bbd7-65643b813d0e-service-ca-bundle\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.996842 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/8035ce1c-4e18-4f71-bbd7-65643b813d0e-snapshots\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.996885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.996855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8035ce1c-4e18-4f71-bbd7-65643b813d0e-tmp\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.997151 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.997130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f6687a09-2cda-40ca-8023-71593b98bf54-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.997382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.997363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/05b2274a-8be6-4170-9723-81ff3601f471-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:26.997449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.997436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8035ce1c-4e18-4f71-bbd7-65643b813d0e-trusted-ca-bundle\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:26.998327 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.998310    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6687a09-2cda-40ca-8023-71593b98bf54-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:26.998440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:26.998426    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8035ce1c-4e18-4f71-bbd7-65643b813d0e-serving-cert\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:27.004962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.004944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrs9q\" (UniqueName: \"kubernetes.io/projected/f6687a09-2cda-40ca-8023-71593b98bf54-kube-api-access-zrs9q\") pod \"kube-storage-version-migrator-operator-6c8f84f897-mbn7j\" (UID: \"f6687a09-2cda-40ca-8023-71593b98bf54\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:27.005446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.005420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnl8m\" (UniqueName: \"kubernetes.io/projected/8035ce1c-4e18-4f71-bbd7-65643b813d0e-kube-api-access-lnl8m\") pod \"insights-operator-5844b856db-jkm2r\" (UID: \"8035ce1c-4e18-4f71-bbd7-65643b813d0e\") " pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:27.005582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.005562    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:27.006482 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.006466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp6xb\" (UniqueName: \"kubernetes.io/projected/05b2274a-8be6-4170-9723-81ff3601f471-kube-api-access-tp6xb\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:27.014281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.014250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"
Sep 11 18:11:27.097545 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.097515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccl56\" (UniqueName: \"kubernetes.io/projected/fb828f6e-099c-437c-9501-7f9e26941795-kube-api-access-ccl56\") pod \"network-check-source-6f8b768455-4n827\" (UID: \"fb828f6e-099c-437c-9501-7f9e26941795\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827"
Sep 11 18:11:27.097719 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.097700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7060fb14-474d-48f5-9e21-9067715c8951-config\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.097758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.097731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7060fb14-474d-48f5-9e21-9067715c8951-serving-cert\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.097758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.097749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdncn\" (UniqueName: \"kubernetes.io/projected/7060fb14-474d-48f5-9e21-9067715c8951-kube-api-access-gdncn\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.098239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.098222    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7060fb14-474d-48f5-9e21-9067715c8951-config\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.099757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.099740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7060fb14-474d-48f5-9e21-9067715c8951-serving-cert\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.106154 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.106133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdncn\" (UniqueName: \"kubernetes.io/projected/7060fb14-474d-48f5-9e21-9067715c8951-kube-api-access-gdncn\") pod \"service-ca-operator-cf7c776b7-pxfqf\" (UID: \"7060fb14-474d-48f5-9e21-9067715c8951\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.106389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.106372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccl56\" (UniqueName: \"kubernetes.io/projected/fb828f6e-099c-437c-9501-7f9e26941795-kube-api-access-ccl56\") pod \"network-check-source-6f8b768455-4n827\" (UID: \"fb828f6e-099c-437c-9501-7f9e26941795\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827"
Sep 11 18:11:27.130118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.130088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"
Sep 11 18:11:27.142763 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.142748    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-jkm2r"
Sep 11 18:11:27.169758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.169725    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:27.239021 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.238962    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827"
Sep 11 18:11:27.252152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.252132    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"
Sep 11 18:11:27.281723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.281687    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n"]
Sep 11 18:11:27.284278 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:27.284235    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ba54c6_8c8f_4fba_9954_b6e3cd3f9fb2.slice/crio-d6fefbdb7a6099369ea76e6415abe8efaa70f6e019c7ea23d4f55bd105d27618 WatchSource:0}: Error finding container d6fefbdb7a6099369ea76e6415abe8efaa70f6e019c7ea23d4f55bd105d27618: Status 404 returned error can't find the container with id d6fefbdb7a6099369ea76e6415abe8efaa70f6e019c7ea23d4f55bd105d27618
Sep 11 18:11:27.412976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.412954    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j"]
Sep 11 18:11:27.414278 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:27.414240    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6687a09_2cda_40ca_8023_71593b98bf54.slice/crio-88927a88c8bb85d23bf901dfe245523665a289ac201cfa9aaf2468253a21558e WatchSource:0}: Error finding container 88927a88c8bb85d23bf901dfe245523665a289ac201cfa9aaf2468253a21558e: Status 404 returned error can't find the container with id 88927a88c8bb85d23bf901dfe245523665a289ac201cfa9aaf2468253a21558e
Sep 11 18:11:27.431336 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.431314    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-jkm2r"]
Sep 11 18:11:27.433890 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:27.433869    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8035ce1c_4e18_4f71_bbd7_65643b813d0e.slice/crio-20530b4750efd338f2f2d076860ae8c29af281cb89170e9c5b47489851fe21e5 WatchSource:0}: Error finding container 20530b4750efd338f2f2d076860ae8c29af281cb89170e9c5b47489851fe21e5: Status 404 returned error can't find the container with id 20530b4750efd338f2f2d076860ae8c29af281cb89170e9c5b47489851fe21e5
Sep 11 18:11:27.502248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.502201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:27.502371 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:27.502351    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:27.502417 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:27.502406    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls podName:05b2274a-8be6-4170-9723-81ff3601f471 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:28.502390779 +0000 UTC m=+57.095591329 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6hwgd" (UID: "05b2274a-8be6-4170-9723-81ff3601f471") : secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:27.510579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.510551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-4n827"]
Sep 11 18:11:27.512784 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:27.512764    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb828f6e_099c_437c_9501_7f9e26941795.slice/crio-4f97bab547318ef3998b8df8745fb32bb0a3381982f030fdc83454e7a9ceed7b WatchSource:0}: Error finding container 4f97bab547318ef3998b8df8745fb32bb0a3381982f030fdc83454e7a9ceed7b: Status 404 returned error can't find the container with id 4f97bab547318ef3998b8df8745fb32bb0a3381982f030fdc83454e7a9ceed7b
Sep 11 18:11:27.532276 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.532229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf"]
Sep 11 18:11:27.535826 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:27.535804    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7060fb14_474d_48f5_9e21_9067715c8951.slice/crio-bd3d3e740cecba0f18244955711147014845765cb3735dee48ffdf68025f33a6 WatchSource:0}: Error finding container bd3d3e740cecba0f18244955711147014845765cb3735dee48ffdf68025f33a6: Status 404 returned error can't find the container with id bd3d3e740cecba0f18244955711147014845765cb3735dee48ffdf68025f33a6
Sep 11 18:11:27.590855 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.590836    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67f795677b-ck2df"]
Sep 11 18:11:27.593579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.593563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.597594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.597577    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 18:11:27.597688 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.597575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ll578\""
Sep 11 18:11:27.597688 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.597595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 18:11:27.597924 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.597910    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 18:11:27.602519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-bound-sa-token\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-trusted-ca\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-image-registry-private-configuration\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvjpx\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-kube-api-access-mvjpx\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-certificates\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a41f3cc4-74fe-419c-862c-8d120a60bf51-ca-trust-extracted\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.602757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.602741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-installation-pull-secrets\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.603215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.603197    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 18:11:27.614183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.614165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f795677b-ck2df"]
Sep 11 18:11:27.703354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-bound-sa-token\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-trusted-ca\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-image-registry-private-configuration\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvjpx\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-kube-api-access-mvjpx\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-certificates\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a41f3cc4-74fe-419c-862c-8d120a60bf51-ca-trust-extracted\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:27.703563    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:27.703580    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f795677b-ck2df: secret "image-registry-tls" not found
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-installation-pull-secrets\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.703734 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:27.703628    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls podName:a41f3cc4-74fe-419c-862c-8d120a60bf51 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:28.203609226 +0000 UTC m=+56.796809762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls") pod "image-registry-67f795677b-ck2df" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51") : secret "image-registry-tls" not found
Sep 11 18:11:27.704235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.703855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a41f3cc4-74fe-419c-862c-8d120a60bf51-ca-trust-extracted\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.704235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.704063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-certificates\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.704369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.704347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-trusted-ca\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.705697 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.705676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-installation-pull-secrets\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.705771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.705727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-image-registry-private-configuration\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.713886 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.713869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-bound-sa-token\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:27.714421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:27.714405    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvjpx\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-kube-api-access-mvjpx\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:28.207072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.207036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:28.207356 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:28.207293    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 18:11:28.207356 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:28.207311    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f795677b-ck2df: secret "image-registry-tls" not found
Sep 11 18:11:28.207470 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:28.207368    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls podName:a41f3cc4-74fe-419c-862c-8d120a60bf51 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:29.207349261 +0000 UTC m=+57.800549800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls") pod "image-registry-67f795677b-ck2df" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51") : secret "image-registry-tls" not found
Sep 11 18:11:28.263547 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.263445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j" event={"ID":"f6687a09-2cda-40ca-8023-71593b98bf54","Type":"ContainerStarted","Data":"88927a88c8bb85d23bf901dfe245523665a289ac201cfa9aaf2468253a21558e"}
Sep 11 18:11:28.267985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.267953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n" event={"ID":"72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2","Type":"ContainerStarted","Data":"d6fefbdb7a6099369ea76e6415abe8efaa70f6e019c7ea23d4f55bd105d27618"}
Sep 11 18:11:28.270160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.270129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827" event={"ID":"fb828f6e-099c-437c-9501-7f9e26941795","Type":"ContainerStarted","Data":"4f97bab547318ef3998b8df8745fb32bb0a3381982f030fdc83454e7a9ceed7b"}
Sep 11 18:11:28.272911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.272859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf" event={"ID":"7060fb14-474d-48f5-9e21-9067715c8951","Type":"ContainerStarted","Data":"bd3d3e740cecba0f18244955711147014845765cb3735dee48ffdf68025f33a6"}
Sep 11 18:11:28.274798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.274773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jkm2r" event={"ID":"8035ce1c-4e18-4f71-bbd7-65643b813d0e","Type":"ContainerStarted","Data":"20530b4750efd338f2f2d076860ae8c29af281cb89170e9c5b47489851fe21e5"}
Sep 11 18:11:28.509791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:28.509706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:28.509963 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:28.509894    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:28.509963 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:28.509956    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls podName:05b2274a-8be6-4170-9723-81ff3601f471 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:30.509939622 +0000 UTC m=+59.103140164 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6hwgd" (UID: "05b2274a-8be6-4170-9723-81ff3601f471") : secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:29.216575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:29.216536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:29.217029 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:29.216723    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 18:11:29.217029 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:29.216745    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f795677b-ck2df: secret "image-registry-tls" not found
Sep 11 18:11:29.217029 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:29.216813    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls podName:a41f3cc4-74fe-419c-862c-8d120a60bf51 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:31.216792398 +0000 UTC m=+59.809992960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls") pod "image-registry-67f795677b-ck2df" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51") : secret "image-registry-tls" not found
Sep 11 18:11:30.527834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:30.527797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:30.528339 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:30.527995    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:30.528339 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:30.528057    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls podName:05b2274a-8be6-4170-9723-81ff3601f471 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:34.528042476 +0000 UTC m=+63.121243010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6hwgd" (UID: "05b2274a-8be6-4170-9723-81ff3601f471") : secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:31.191048 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:31.191016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7bpkd"
Sep 11 18:11:31.233690 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:31.233660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:31.233872 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:31.233776    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 18:11:31.233872 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:31.233790    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f795677b-ck2df: secret "image-registry-tls" not found
Sep 11 18:11:31.233872 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:31.233841    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls podName:a41f3cc4-74fe-419c-862c-8d120a60bf51 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:35.233824029 +0000 UTC m=+63.827024564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls") pod "image-registry-67f795677b-ck2df" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51") : secret "image-registry-tls" not found
Sep 11 18:11:31.324038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:31.324011    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:31.388550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:31.388520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:32.288772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.288730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf" event={"ID":"7060fb14-474d-48f5-9e21-9067715c8951","Type":"ContainerStarted","Data":"6b5c5751fcf0f39773c2b90141c36739d5ded11fae4bf7b516c9bf38d61dbb88"}
Sep 11 18:11:32.290764 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.290731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jkm2r" event={"ID":"8035ce1c-4e18-4f71-bbd7-65643b813d0e","Type":"ContainerStarted","Data":"1831730a0ce65ed25b8efa7f351fa4b06eaba119d51c84b1542c46caf851aacc"}
Sep 11 18:11:32.293028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.293000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j" event={"ID":"f6687a09-2cda-40ca-8023-71593b98bf54","Type":"ContainerStarted","Data":"b89527e495e8ca069cde9d08204b1ca812233a02ff4a902a337fb340bf303cf4"}
Sep 11 18:11:32.294558 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.294534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n" event={"ID":"72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2","Type":"ContainerStarted","Data":"b760aacff30c2752c4918009a10018154a5132d7c36b54bf40d32e9885d69ba1"}
Sep 11 18:11:32.296114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.296082    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827" event={"ID":"fb828f6e-099c-437c-9501-7f9e26941795","Type":"ContainerStarted","Data":"16c4b660d1677630f369e7e05ed51fce5eb11ac12d40fff7d3faf3f141e1d7aa"}
Sep 11 18:11:32.307755 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.307626    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf" podStartSLOduration=2.072287898 podStartE2EDuration="6.307610373s" totalImagesPullingTime="4.235322475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:32.307387884 +0000 UTC m=+60.900588454" watchObservedRunningTime="2026-09-11 18:11:32.307610373 +0000 UTC m=+60.900810927"
Sep 11 18:11:32.323854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.323806    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-vtq6n" podStartSLOduration=1.849605942 podStartE2EDuration="6.323788526s" totalImagesPullingTime="4.474182584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:32.322430548 +0000 UTC m=+60.915631107" watchObservedRunningTime="2026-09-11 18:11:32.323788526 +0000 UTC m=+60.916989084"
Sep 11 18:11:32.342819 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.342769    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-4n827" podStartSLOduration=2.078595018 podStartE2EDuration="6.342753411s" totalImagesPullingTime="4.264158393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:32.342374227 +0000 UTC m=+60.935574782" watchObservedRunningTime="2026-09-11 18:11:32.342753411 +0000 UTC m=+60.935953971"
Sep 11 18:11:32.369106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.369065    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j" podStartSLOduration=2.014590315 podStartE2EDuration="6.369048483s" totalImagesPullingTime="4.354458168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:32.367757919 +0000 UTC m=+60.960958481" watchObservedRunningTime="2026-09-11 18:11:32.369048483 +0000 UTC m=+60.962249041"
Sep 11 18:11:32.388029 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.387989    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-jkm2r" podStartSLOduration=2.05630719 podStartE2EDuration="6.387972979s" totalImagesPullingTime="4.331665789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:32.387705833 +0000 UTC m=+60.980906392" watchObservedRunningTime="2026-09-11 18:11:32.387972979 +0000 UTC m=+60.981173537"
Sep 11 18:11:32.408834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.408807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:32.508684 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:32.508648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:33.663171 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.663129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:33.801976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.801940    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:33.860100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.860073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8v4wp"]
Sep 11 18:11:33.862766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.862748    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:33.864891 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.864867    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 18:11:33.864992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.864970    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 18:11:33.864992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.864985    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8hchm\""
Sep 11 18:11:33.876139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.876119    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8v4wp"]
Sep 11 18:11:33.896294 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.896272    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:33.959478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.959423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/006f2d67-3840-4a1b-b6b8-7ead931c73c2-data-volume\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:33.959571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.959488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:33.959571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.959521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:33.959662 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.959583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/006f2d67-3840-4a1b-b6b8-7ead931c73c2-crio-socket\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:33.959662 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:33.959611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t7w5\" (UniqueName: \"kubernetes.io/projected/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-api-access-6t7w5\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/006f2d67-3840-4a1b-b6b8-7ead931c73c2-data-volume\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060340 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060340 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060444 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.060415    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:11:34.060444 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.060430    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:34.060525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/006f2d67-3840-4a1b-b6b8-7ead931c73c2-crio-socket\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060525 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.060481    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:34.560465307 +0000 UTC m=+63.153665843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:34.060525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/006f2d67-3840-4a1b-b6b8-7ead931c73c2-crio-socket\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060660 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.060531    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:34.560510398 +0000 UTC m=+63.153710946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : configmap "kube-rbac-proxy" not found
Sep 11 18:11:34.060660 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/006f2d67-3840-4a1b-b6b8-7ead931c73c2-data-volume\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.060660 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.060612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t7w5\" (UniqueName: \"kubernetes.io/projected/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-api-access-6t7w5\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.081841 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.081817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t7w5\" (UniqueName: \"kubernetes.io/projected/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-api-access-6t7w5\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.385504 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.385481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:34.477460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.477435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:34.564834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.564808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.564913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.564890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:34.564954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:34.564912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:34.565004 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.564950    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:34.565004 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.564980    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:11:34.565075 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.565010    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:35.564997045 +0000 UTC m=+64.158197579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:34.565075 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.565018    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:34.565075 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.565029    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:35.565017401 +0000 UTC m=+64.158217936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : configmap "kube-rbac-proxy" not found
Sep 11 18:11:34.565075 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:34.565067    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls podName:05b2274a-8be6-4170-9723-81ff3601f471 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:42.565054934 +0000 UTC m=+71.158255468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6hwgd" (UID: "05b2274a-8be6-4170-9723-81ff3601f471") : secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:35.269501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.269469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:35.269826 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.269570    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 18:11:35.269826 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.269582    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f795677b-ck2df: secret "image-registry-tls" not found
Sep 11 18:11:35.269826 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.269626    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls podName:a41f3cc4-74fe-419c-862c-8d120a60bf51 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:43.269613331 +0000 UTC m=+71.862813880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls") pod "image-registry-67f795677b-ck2df" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51") : secret "image-registry-tls" not found
Sep 11 18:11:35.571700 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.571670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:35.571854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.571714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:35.571854 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.571798    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:11:35.571854 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.571825    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:35.572013 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.571871    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:37.571850817 +0000 UTC m=+66.165051375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : configmap "kube-rbac-proxy" not found
Sep 11 18:11:35.572013 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:35.571895    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:37.571885053 +0000 UTC m=+66.165085591 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:35.938396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.938337    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-sbrrt"]
Sep 11 18:11:35.971693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.971676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-sbrrt"]
Sep 11 18:11:35.971797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.971750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:35.974017 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.973986    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 18:11:35.974017 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.973996    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 18:11:35.974017 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.974014    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 18:11:35.974209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.974085    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 18:11:35.974847 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.974832    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-np7k4\""
Sep 11 18:11:35.982852 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:35.982833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:36.058955 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.058930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:36.075041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.075016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/93da0e9e-6097-4962-bf64-2a1c48e70063-signing-cabundle\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.075159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.075142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/93da0e9e-6097-4962-bf64-2a1c48e70063-signing-key\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.075209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.075193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7gps\" (UniqueName: \"kubernetes.io/projected/93da0e9e-6097-4962-bf64-2a1c48e70063-kube-api-access-z7gps\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.175946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.175923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/93da0e9e-6097-4962-bf64-2a1c48e70063-signing-key\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.176048 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.175975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7gps\" (UniqueName: \"kubernetes.io/projected/93da0e9e-6097-4962-bf64-2a1c48e70063-kube-api-access-z7gps\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.176048 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.176010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/93da0e9e-6097-4962-bf64-2a1c48e70063-signing-cabundle\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.176684 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.176664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/93da0e9e-6097-4962-bf64-2a1c48e70063-signing-cabundle\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.178302 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.178283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/93da0e9e-6097-4962-bf64-2a1c48e70063-signing-key\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.185041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.185017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7gps\" (UniqueName: \"kubernetes.io/projected/93da0e9e-6097-4962-bf64-2a1c48e70063-kube-api-access-z7gps\") pod \"service-ca-568869fdc5-sbrrt\" (UID: \"93da0e9e-6097-4962-bf64-2a1c48e70063\") " pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.281137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.281115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-sbrrt"
Sep 11 18:11:36.475129 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.475097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:36.580448 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.580423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-sbrrt"]
Sep 11 18:11:36.581744 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:36.581717    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93da0e9e_6097_4962_bf64_2a1c48e70063.slice/crio-6add4e523cf0dce532234ca2aabeee3fe74bb8f900fc754b3b1a88ac1fba8161 WatchSource:0}: Error finding container 6add4e523cf0dce532234ca2aabeee3fe74bb8f900fc754b3b1a88ac1fba8161: Status 404 returned error can't find the container with id 6add4e523cf0dce532234ca2aabeee3fe74bb8f900fc754b3b1a88ac1fba8161
Sep 11 18:11:36.592678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.592659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:36.790646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:36.790622    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x46v2_3b656de9-f949-4ae4-b1a7-9ee77032b734/dns-node-resolver/0.log"
Sep 11 18:11:37.312998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.312959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-sbrrt" event={"ID":"93da0e9e-6097-4962-bf64-2a1c48e70063","Type":"ContainerStarted","Data":"c4a63f0584811ccb622734a33a51448318f7151d56813c2a2e8c8f6ddc74446a"}
Sep 11 18:11:37.313377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.313020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-sbrrt" event={"ID":"93da0e9e-6097-4962-bf64-2a1c48e70063","Type":"ContainerStarted","Data":"6add4e523cf0dce532234ca2aabeee3fe74bb8f900fc754b3b1a88ac1fba8161"}
Sep 11 18:11:37.338146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.338100    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-sbrrt" podStartSLOduration=2.338086956 podStartE2EDuration="2.338086956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:37.336596039 +0000 UTC m=+65.929796593" watchObservedRunningTime="2026-09-11 18:11:37.338086956 +0000 UTC m=+65.931287512"
Sep 11 18:11:37.382424 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.382400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:37.388954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.388937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ft7z_117de80d-dc8d-43d8-b406-85c3a636ecb4/node-ca/0.log"
Sep 11 18:11:37.588401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.588347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:11:37.588401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.588394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:37.588521 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.588454    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:11:37.588521 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.588493    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:41.58848273 +0000 UTC m=+70.181683277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : configmap "kube-rbac-proxy" not found
Sep 11 18:11:37.588521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.588513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:37.588650 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.588533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:11:37.588650 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.588613    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:37.588650 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.588632    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 18:11:37.588741 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.588669    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:41.58865193 +0000 UTC m=+70.181852475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:37.588741 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.588686    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls podName:6b50bc98-cf61-4994-bcfd-9c43af9229e4 nodeName:}" failed. No retries permitted until 2026-09-11 18:12:09.588677931 +0000 UTC m=+98.181878466 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls") pod "dns-default-vk549" (UID: "6b50bc98-cf61-4994-bcfd-9c43af9229e4") : secret "dns-default-metrics-tls" not found
Sep 11 18:11:37.590842 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.590825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 18:11:37.599191 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.599172    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 18:11:37.599298 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:37.599224    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs podName:df55e017-4726-49f9-8507-41222acd67f0 nodeName:}" failed. No retries permitted until 2026-09-11 18:12:41.599207704 +0000 UTC m=+130.192408254 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs") pod "network-metrics-daemon-qhmz2" (UID: "df55e017-4726-49f9-8507-41222acd67f0") : secret "metrics-daemon-secret" not found
Sep 11 18:11:37.689367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.689343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:37.689435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.689387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:37.691695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.691676    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 18:11:37.691856 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.691840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwdck\" (UniqueName: \"kubernetes.io/projected/f65abfcd-9d97-4224-958d-f4aeb2d78668-kube-api-access-fwdck\") pod \"network-check-target-m67bj\" (UID: \"f65abfcd-9d97-4224-958d-f4aeb2d78668\") " pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:37.701701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.701683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f2089a3-9817-4c29-980b-f4018a7b145e-original-pull-secret\") pod \"global-pull-secret-syncer-6ntmn\" (UID: \"5f2089a3-9817-4c29-980b-f4018a7b145e\") " pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:37.721130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.721108    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ztbbx\""
Sep 11 18:11:37.723433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.723418    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ntmn"
Sep 11 18:11:37.729706 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.729691    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:37.971498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:37.971421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:38.002547 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.002514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6ntmn"]
Sep 11 18:11:38.005967 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:38.005937    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2089a3_9817_4c29_980b_f4018a7b145e.slice/crio-218a91a55ffe33e6f992a49a7e40d3efdcb62720c9ec80ca710d41e74900b200 WatchSource:0}: Error finding container 218a91a55ffe33e6f992a49a7e40d3efdcb62720c9ec80ca710d41e74900b200: Status 404 returned error can't find the container with id 218a91a55ffe33e6f992a49a7e40d3efdcb62720c9ec80ca710d41e74900b200
Sep 11 18:11:38.009541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.009176    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m67bj"]
Sep 11 18:11:38.319749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.319017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m67bj" event={"ID":"f65abfcd-9d97-4224-958d-f4aeb2d78668","Type":"ContainerStarted","Data":"76d1f64e0004ef524e70d8ed33c3911a5adde50526f578cfaa3bcd2deb4f26b3"}
Sep 11 18:11:38.319749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.319067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m67bj" event={"ID":"f65abfcd-9d97-4224-958d-f4aeb2d78668","Type":"ContainerStarted","Data":"e140dad52eee2342fa49ec0e14b05c407ded79462ed35dcdfd3400183330d222"}
Sep 11 18:11:38.319749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.319699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:11:38.321467 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.321444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6ntmn" event={"ID":"5f2089a3-9817-4c29-980b-f4018a7b145e","Type":"ContainerStarted","Data":"218a91a55ffe33e6f992a49a7e40d3efdcb62720c9ec80ca710d41e74900b200"}
Sep 11 18:11:38.337537 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.337497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m67bj" podStartSLOduration=66.337485084 podStartE2EDuration="1m6.337485084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:38.337208917 +0000 UTC m=+66.930409475" watchObservedRunningTime="2026-09-11 18:11:38.337485084 +0000 UTC m=+66.930685640"
Sep 11 18:11:38.356922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.356893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:38.390746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.390725    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rjkqn_0266d274-0277-49ea-ac36-9e8bf474fae3/serve-healthcheck-canary/0.log"
Sep 11 18:11:38.417567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.417533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:38.488901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.488877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:38.589853 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.589756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:38.739414 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:38.739332    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:39.191102 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:39.191075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-mbn7j_f6687a09-2cda-40ca-8023-71593b98bf54/kube-storage-version-migrator-operator/0.log"
Sep 11 18:11:39.346652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:39.346620    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:39.950492 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:39.950456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:40.144638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:40.144611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:40.340478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:40.340444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:40.882659 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:40.882624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.147506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:41.147386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.543071 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:41.543041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.628507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:41.628456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:41.628689 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:41.628512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:41.628689 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:41.628647    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 18:11:41.628689 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:41.628663    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:41.628845 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:41.628726    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:49.628703818 +0000 UTC m=+78.221904357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : configmap "kube-rbac-proxy" not found
Sep 11 18:11:41.628845 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:41.628745    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls podName:006f2d67-3840-4a1b-b6b8-7ead931c73c2 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:49.628736018 +0000 UTC m=+78.221936557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8v4wp" (UID: "006f2d67-3840-4a1b-b6b8-7ead931c73c2") : secret "insights-runtime-extractor-tls" not found
Sep 11 18:11:41.853669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:41.853588    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.954839 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:41.954753    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:42.057964 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:42.057935    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:42.157396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:42.157313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:42.256477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:42.256446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:42.542288 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:42.542235    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:42.635747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:42.635715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:42.635897 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:42.635877    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:42.635964 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:11:42.635954    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls podName:05b2274a-8be6-4170-9723-81ff3601f471 nodeName:}" failed. No retries permitted until 2026-09-11 18:11:58.635933911 +0000 UTC m=+87.229134450 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-6hwgd" (UID: "05b2274a-8be6-4170-9723-81ff3601f471") : secret "cluster-monitoring-operator-tls" not found
Sep 11 18:11:42.941661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:42.941634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:43.216632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.216559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:43.340868 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.340845    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:43.343132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.343100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"image-registry-67f795677b-ck2df\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") " pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:43.344063 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.344037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:43.347612 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.347556    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6ntmn" event={"ID":"5f2089a3-9817-4c29-980b-f4018a7b145e","Type":"ContainerStarted","Data":"79e23dfec7c5c885aef7a12d61deb592cf8b709636cb2797d74b6902b4e330e8"}
Sep 11 18:11:43.365853 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.365819    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6ntmn" podStartSLOduration=67.059079683 podStartE2EDuration="1m11.365803549s" totalImagesPullingTime="4.306723866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:43.365403097 +0000 UTC m=+71.958603655" watchObservedRunningTime="2026-09-11 18:11:43.365803549 +0000 UTC m=+71.959004107"
Sep 11 18:11:43.504349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.504290    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ll578\""
Sep 11 18:11:43.512246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.512225    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:43.802759 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:43.802725    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f795677b-ck2df"]
Sep 11 18:11:43.806735 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:43.806711    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda41f3cc4_74fe_419c_862c_8d120a60bf51.slice/crio-98491ac8f8389a101671b832e03747ae67caf3e3030c87b483005527901ab4e2 WatchSource:0}: Error finding container 98491ac8f8389a101671b832e03747ae67caf3e3030c87b483005527901ab4e2: Status 404 returned error can't find the container with id 98491ac8f8389a101671b832e03747ae67caf3e3030c87b483005527901ab4e2
Sep 11 18:11:44.148928 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.148847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:44.344088 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.344039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:44.351650 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.351622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f795677b-ck2df" event={"ID":"a41f3cc4-74fe-419c-862c-8d120a60bf51","Type":"ContainerStarted","Data":"c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6"}
Sep 11 18:11:44.351792 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.351676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f795677b-ck2df" event={"ID":"a41f3cc4-74fe-419c-862c-8d120a60bf51","Type":"ContainerStarted","Data":"98491ac8f8389a101671b832e03747ae67caf3e3030c87b483005527901ab4e2"}
Sep 11 18:11:44.374526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.374476    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67f795677b-ck2df" podStartSLOduration=17.374458081 podStartE2EDuration="17.374458081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:44.373298231 +0000 UTC m=+72.966498790" watchObservedRunningTime="2026-09-11 18:11:44.374458081 +0000 UTC m=+72.967658639"
Sep 11 18:11:44.475553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.475490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:44.755767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.755706    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:44.947724 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:44.947699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:45.355026 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:45.355002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:45.355449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:45.355124    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:11:45.977367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:45.977338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:46.551618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:46.551591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:46.745180 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:46.745151    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:46.843946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:46.843886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:47.150127 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:47.150071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:47.348190 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:47.348166    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:47.744684 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:47.744655    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:48.146999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:48.146978    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:48.338038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:48.338012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:48.750516 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:48.750487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:48.816433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:48.816401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:49.591770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.591737    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:49.689931 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.689902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:49.690043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.689981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:49.690459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.690424    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/006f2d67-3840-4a1b-b6b8-7ead931c73c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:49.692293 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.692255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/006f2d67-3840-4a1b-b6b8-7ead931c73c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8v4wp\" (UID: \"006f2d67-3840-4a1b-b6b8-7ead931c73c2\") " pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:49.694889 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.694864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:49.770615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.770582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:49.771477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.771461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8v4wp"
Sep 11 18:11:49.962617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:49.962562    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:50.067455 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.067436    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8v4wp"]
Sep 11 18:11:50.069576 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:50.069547    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006f2d67_3840_4a1b_b6b8_7ead931c73c2.slice/crio-616733218f814a5effd87b582998641d6021649ec38959342fbc138622a0fd1d WatchSource:0}: Error finding container 616733218f814a5effd87b582998641d6021649ec38959342fbc138622a0fd1d: Status 404 returned error can't find the container with id 616733218f814a5effd87b582998641d6021649ec38959342fbc138622a0fd1d
Sep 11 18:11:50.278204 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.278174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:50.355580 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.355554    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:50.371295 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.371271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8v4wp" event={"ID":"006f2d67-3840-4a1b-b6b8-7ead931c73c2","Type":"ContainerStarted","Data":"f2f3bb9f259d160adfa0cd8035f333b7ab72089385932d4524935b9f36228456"}
Sep 11 18:11:50.371378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.371326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8v4wp" event={"ID":"006f2d67-3840-4a1b-b6b8-7ead931c73c2","Type":"ContainerStarted","Data":"616733218f814a5effd87b582998641d6021649ec38959342fbc138622a0fd1d"}
Sep 11 18:11:50.471774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.471751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:50.540838 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.540783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:50.900251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:50.900223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:51.008071 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:51.008044    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:51.282915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:51.282886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:51.376887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:51.376855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8v4wp" event={"ID":"006f2d67-3840-4a1b-b6b8-7ead931c73c2","Type":"ContainerStarted","Data":"994eaa0698a850bfb4c3768fbc7f63b93efc4eee292dc7263bf2c1b0f0f4fe28"}
Sep 11 18:11:51.494488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:51.494450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:51.660629 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:51.660548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:51.728942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:51.728909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:52.116830 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:52.116807    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:52.216333 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:52.216302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:52.382086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:52.382024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8v4wp" event={"ID":"006f2d67-3840-4a1b-b6b8-7ead931c73c2","Type":"ContainerStarted","Data":"5b86affa447bb069cca49b9895188a3cb826ce2d438e2bfa43a6df3417126eb1"}
Sep 11 18:11:52.411946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:52.411884    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8v4wp" podStartSLOduration=17.489142427 podStartE2EDuration="19.411868518s" totalImagesPullingTime="1.922726091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:11:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:52.411155093 +0000 UTC m=+81.004355649" watchObservedRunningTime="2026-09-11 18:11:52.411868518 +0000 UTC m=+81.005069072"
Sep 11 18:11:55.954466 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:55.954356    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:56.055563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:56.055532    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:58.649854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:58.649810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:58.652304 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:58.652283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/05b2274a-8be6-4170-9723-81ff3601f471-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-6hwgd\" (UID: \"05b2274a-8be6-4170-9723-81ff3601f471\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:58.951146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:58.951069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dj9fm\""
Sep 11 18:11:58.959163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:58.959141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"
Sep 11 18:11:59.237696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:59.237672    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd"]
Sep 11 18:11:59.240131 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:11:59.240104    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b2274a_8be6_4170_9723_81ff3601f471.slice/crio-fd888238b5a10541f78e27da0e3957a02168a04883f6994c2d7f86398b215ea7 WatchSource:0}: Error finding container fd888238b5a10541f78e27da0e3957a02168a04883f6994c2d7f86398b215ea7: Status 404 returned error can't find the container with id fd888238b5a10541f78e27da0e3957a02168a04883f6994c2d7f86398b215ea7
Sep 11 18:11:59.405990 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:11:59.405963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd" event={"ID":"05b2274a-8be6-4170-9723-81ff3601f471","Type":"ContainerStarted","Data":"fd888238b5a10541f78e27da0e3957a02168a04883f6994c2d7f86398b215ea7"}
Sep 11 18:12:00.580452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.580423    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f795677b-ck2df"]
Sep 11 18:12:00.585116 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.585080    2563 patch_prober.go:28] interesting pod/image-registry-67f795677b-ck2df container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 18:12:00.585303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.585131    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-67f795677b-ck2df" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:12:00.690582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.690548    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db649f896-wzjc8"]
Sep 11 18:12:00.695321 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.695299    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.713556 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.713526    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db649f896-wzjc8"]
Sep 11 18:12:00.756187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.756163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:00.762649 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-ca-trust-extracted\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.762767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-installation-pull-secrets\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.762767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q888\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-kube-api-access-9q888\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.762767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-bound-sa-token\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.762900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-image-registry-private-configuration\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.762900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-registry-tls\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.762900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-trusted-ca\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.763045 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.762903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-registry-certificates\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.857015 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.856943    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:00.863632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-image-registry-private-configuration\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-registry-tls\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-trusted-ca\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-registry-certificates\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-ca-trust-extracted\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863752    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-installation-pull-secrets\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q888\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-kube-api-access-9q888\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.863988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.863853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-bound-sa-token\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.864462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.864432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-ca-trust-extracted\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.865167 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.865124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-registry-certificates\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.865311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.865226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-trusted-ca\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.866482 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.866461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-image-registry-private-configuration\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.866591 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.866571    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-registry-tls\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.866910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.866889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-installation-pull-secrets\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.875228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.875205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-bound-sa-token\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:00.877611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:00.877589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q888\" (UniqueName: \"kubernetes.io/projected/37c05e10-e9f0-4f5b-beee-8617ab2ffc96-kube-api-access-9q888\") pod \"image-registry-db649f896-wzjc8\" (UID: \"37c05e10-e9f0-4f5b-beee-8617ab2ffc96\") " pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:01.004968 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.004928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:01.336272 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.336210    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db649f896-wzjc8"]
Sep 11 18:12:01.338217 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:01.338187    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c05e10_e9f0_4f5b_beee_8617ab2ffc96.slice/crio-c36012b4a4e507285c13a7a69f0f7b4211c4fd01df02e83656e785107ad14fe9 WatchSource:0}: Error finding container c36012b4a4e507285c13a7a69f0f7b4211c4fd01df02e83656e785107ad14fe9: Status 404 returned error can't find the container with id c36012b4a4e507285c13a7a69f0f7b4211c4fd01df02e83656e785107ad14fe9
Sep 11 18:12:01.416396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.416370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd" event={"ID":"05b2274a-8be6-4170-9723-81ff3601f471","Type":"ContainerStarted","Data":"4bf5c0c98d8ef6971ed3d121d7013426d1256f2c4079179b2750e3062c91550c"}
Sep 11 18:12:01.418185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.418158    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db649f896-wzjc8" event={"ID":"37c05e10-e9f0-4f5b-beee-8617ab2ffc96","Type":"ContainerStarted","Data":"e7a45220221bf3878441e35a9f0931d68b40bbb329809013f7cefb504170552c"}
Sep 11 18:12:01.418311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.418204    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db649f896-wzjc8" event={"ID":"37c05e10-e9f0-4f5b-beee-8617ab2ffc96","Type":"ContainerStarted","Data":"c36012b4a4e507285c13a7a69f0f7b4211c4fd01df02e83656e785107ad14fe9"}
Sep 11 18:12:01.418311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.418245    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:01.447312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.447248    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-6hwgd" podStartSLOduration=33.579997138 podStartE2EDuration="35.447230929s" totalImagesPullingTime="1.867233791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:01.445725606 +0000 UTC m=+90.038926161" watchObservedRunningTime="2026-09-11 18:12:01.447230929 +0000 UTC m=+90.040431485"
Sep 11 18:12:01.490178 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.490135    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db649f896-wzjc8" podStartSLOduration=1.49012021 podStartE2EDuration="1.49012021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:01.489487465 +0000 UTC m=+90.082688048" watchObservedRunningTime="2026-09-11 18:12:01.49012021 +0000 UTC m=+90.083320766"
Sep 11 18:12:01.543270 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.543241    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:01.709045 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.708965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:01.839579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:01.839553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:02.069419 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:02.069392    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:04.384650 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:04.384613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:05.810729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:05.810690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:05.912578 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:05.912543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:06.102166 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:06.102090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:06.904803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:06.904774    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:06.990211 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:06.990184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:09.305227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.305192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:09.372351 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.372322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:09.629476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.629427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:12:09.631532 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.631506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b50bc98-cf61-4994-bcfd-9c43af9229e4-metrics-tls\") pod \"dns-default-vk549\" (UID: \"6b50bc98-cf61-4994-bcfd-9c43af9229e4\") " pod="openshift-dns/dns-default-vk549"
Sep 11 18:12:09.649172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.649157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4zn8w\""
Sep 11 18:12:09.657233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.657215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vk549"
Sep 11 18:12:09.929238 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.929175    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vk549"]
Sep 11 18:12:09.932362 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:09.932339    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b50bc98_cf61_4994_bcfd_9c43af9229e4.slice/crio-3fc5e655f1273923374aab461f3830ed79d52190f4c30e4311da12ab4fe2729f WatchSource:0}: Error finding container 3fc5e655f1273923374aab461f3830ed79d52190f4c30e4311da12ab4fe2729f: Status 404 returned error can't find the container with id 3fc5e655f1273923374aab461f3830ed79d52190f4c30e4311da12ab4fe2729f
Sep 11 18:12:09.980414 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:09.980388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:10.331691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:10.331663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m67bj"
Sep 11 18:12:10.447888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:10.447860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vk549" event={"ID":"6b50bc98-cf61-4994-bcfd-9c43af9229e4","Type":"ContainerStarted","Data":"3fc5e655f1273923374aab461f3830ed79d52190f4c30e4311da12ab4fe2729f"}
Sep 11 18:12:10.586569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:10.586495    2563 patch_prober.go:28] interesting pod/image-registry-67f795677b-ck2df container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 18:12:10.586687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:10.586566    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-67f795677b-ck2df" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:12:12.004848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.004819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.219301 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.219246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.255542 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.255485    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"]
Sep 11 18:12:12.259078 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.259061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.261642 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.261622    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 18:12:12.262429 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.262405    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 18:12:12.262547 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.262468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fk2j2\""
Sep 11 18:12:12.262547 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.262539    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 18:12:12.282546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.282527    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"]
Sep 11 18:12:12.296550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.296530    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"]
Sep 11 18:12:12.300152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.300134    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zmg88"]
Sep 11 18:12:12.300289 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.300271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.302850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.302833    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.303362 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.303345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 18:12:12.303441 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.303396    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hmjwt\""
Sep 11 18:12:12.303956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.303939    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 18:12:12.305142 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.305111    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 18:12:12.305243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.305156    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 18:12:12.305243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.305236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 18:12:12.305483 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.305467    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xkt9g\""
Sep 11 18:12:12.305628 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.305611    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 18:12:12.316956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.316928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.317351 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.317327    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"]
Sep 11 18:12:12.347653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.347627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrxk6\" (UniqueName: \"kubernetes.io/projected/a3c07754-c93c-4a26-a5c7-d8484b9bc358-kube-api-access-wrxk6\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.347743 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.347683    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.347856 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.347839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.347892 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.347869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3c07754-c93c-4a26-a5c7-d8484b9bc358-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.417147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.417118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.448249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.448367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-accelerators-collector-config\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-sys\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-tls\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-wtmp\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448409    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-textfile\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3c07754-c93c-4a26-a5c7-d8484b9bc358-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrxk6\" (UniqueName: \"kubernetes.io/projected/a3c07754-c93c-4a26-a5c7-d8484b9bc358-kube-api-access-wrxk6\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ccc46eb-455f-4156-bab2-cd071622b409-metrics-client-ca\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.448573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdj89\" (UniqueName: \"kubernetes.io/projected/3ccc46eb-455f-4156-bab2-cd071622b409-kube-api-access-tdj89\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.449010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.449010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.449010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-root\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.449010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.449010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.448834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ht7r\" (UniqueName: \"kubernetes.io/projected/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-api-access-6ht7r\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.449010 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:12.448966    2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 18:12:12.449301 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:12.449053    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls podName:a3c07754-c93c-4a26-a5c7-d8484b9bc358 nodeName:}" failed. No retries permitted until 2026-09-11 18:12:12.949026551 +0000 UTC m=+101.542227106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-n9wwr" (UID: "a3c07754-c93c-4a26-a5c7-d8484b9bc358") : secret "openshift-state-metrics-tls" not found
Sep 11 18:12:12.449301 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.449204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3c07754-c93c-4a26-a5c7-d8484b9bc358-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.451098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.451076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.458058 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.458023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrxk6\" (UniqueName: \"kubernetes.io/projected/a3c07754-c93c-4a26-a5c7-d8484b9bc358-kube-api-access-wrxk6\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.459070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.459044    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vk549" event={"ID":"6b50bc98-cf61-4994-bcfd-9c43af9229e4","Type":"ContainerStarted","Data":"09843d16c010c5f17352bf0c5c5352e17eecd0a1f25219035075d572a46eab97"}
Sep 11 18:12:12.459170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.459096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vk549" event={"ID":"6b50bc98-cf61-4994-bcfd-9c43af9229e4","Type":"ContainerStarted","Data":"2170185d210f8cfc322aef0255801f8f515fc7bcea0d47334152c0db421cd982"}
Sep 11 18:12:12.459170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.459161    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vk549"
Sep 11 18:12:12.478531 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.478486    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vk549" podStartSLOduration=65.921830225 podStartE2EDuration="1m7.478469903s" totalImagesPullingTime="1.556639678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:12.477143688 +0000 UTC m=+101.070344246" watchObservedRunningTime="2026-09-11 18:12:12.478469903 +0000 UTC m=+101.071670461"
Sep 11 18:12:12.549662 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.549780 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.549780 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-root\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.549893 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:12.549790    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 11 18:12:12.549893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ht7r\" (UniqueName: \"kubernetes.io/projected/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-api-access-6ht7r\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.549893 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:12.549851    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls podName:24ef2858-8dbc-4a5f-97f7-fe8cc5702fab nodeName:}" failed. No retries permitted until 2026-09-11 18:12:13.049832157 +0000 UTC m=+101.643032709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-6xlf9" (UID: "24ef2858-8dbc-4a5f-97f7-fe8cc5702fab") : secret "kube-state-metrics-tls" not found
Sep 11 18:12:12.549893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-root\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.550100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-accelerators-collector-config\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.549993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-sys\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550019    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-tls\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-wtmp\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-sys\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-textfile\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550294    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ccc46eb-455f-4156-bab2-cd071622b409-metrics-client-ca\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdj89\" (UniqueName: \"kubernetes.io/projected/3ccc46eb-455f-4156-bab2-cd071622b409-kube-api-access-tdj89\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.550708 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.550708 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.550549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-textfile\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.551037 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.551013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ccc46eb-455f-4156-bab2-cd071622b409-metrics-client-ca\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.551161 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.551143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-wtmp\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.551226 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.551174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.551297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.551278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-accelerators-collector-config\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.551894 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.551868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.552801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.552747    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.553646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.553624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3ccc46eb-455f-4156-bab2-cd071622b409-node-exporter-tls\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.553760 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.553731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.561137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.561096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ht7r\" (UniqueName: \"kubernetes.io/projected/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-api-access-6ht7r\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:12.569919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.569529    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdj89\" (UniqueName: \"kubernetes.io/projected/3ccc46eb-455f-4156-bab2-cd071622b409-kube-api-access-tdj89\") pod \"node-exporter-zmg88\" (UID: \"3ccc46eb-455f-4156-bab2-cd071622b409\") " pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.611389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.611358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.622374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.622353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zmg88"
Sep 11 18:12:12.667118 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:12.667091    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccc46eb_455f_4156_bab2_cd071622b409.slice/crio-e20731f93329e841ff224ec01016a8679c5217b93e5de83bbceaab3b169aa298 WatchSource:0}: Error finding container e20731f93329e841ff224ec01016a8679c5217b93e5de83bbceaab3b169aa298: Status 404 returned error can't find the container with id e20731f93329e841ff224ec01016a8679c5217b93e5de83bbceaab3b169aa298
Sep 11 18:12:12.697525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.697502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.941866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.941799    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:12.953840 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:12.953816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:12.953985 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:12.953967    2563 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 18:12:12.954055 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:12.954024    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls podName:a3c07754-c93c-4a26-a5c7-d8484b9bc358 nodeName:}" failed. No retries permitted until 2026-09-11 18:12:13.954011294 +0000 UTC m=+102.547211844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-n9wwr" (UID: "a3c07754-c93c-4a26-a5c7-d8484b9bc358") : secret "openshift-state-metrics-tls" not found
Sep 11 18:12:13.054425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.054402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:13.054804 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:13.054568    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 11 18:12:13.054804 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:13.054637    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls podName:24ef2858-8dbc-4a5f-97f7-fe8cc5702fab nodeName:}" failed. No retries permitted until 2026-09-11 18:12:14.054617723 +0000 UTC m=+102.647818263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls") pod "kube-state-metrics-59fbdd77d-6xlf9" (UID: "24ef2858-8dbc-4a5f-97f7-fe8cc5702fab") : secret "kube-state-metrics-tls" not found
Sep 11 18:12:13.176783 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.176740    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:13.371157 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.371121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:13.462100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.462063    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:13.464039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.464017    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmg88" event={"ID":"3ccc46eb-455f-4156-bab2-cd071622b409","Type":"ContainerStarted","Data":"e20731f93329e841ff224ec01016a8679c5217b93e5de83bbceaab3b169aa298"}
Sep 11 18:12:13.695486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.695404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:13.796750 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.796714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:13.962456 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.962375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:13.965215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:13.965189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3c07754-c93c-4a26-a5c7-d8484b9bc358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-n9wwr\" (UID: \"a3c07754-c93c-4a26-a5c7-d8484b9bc358\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:14.063438 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.063412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:14.065601 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.065572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/24ef2858-8dbc-4a5f-97f7-fe8cc5702fab-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-6xlf9\" (UID: \"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:14.069697 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.069682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"
Sep 11 18:12:14.111155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.111109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"
Sep 11 18:12:14.343756 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.343735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr"]
Sep 11 18:12:14.345400 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:14.345379    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c07754_c93c_4a26_a5c7_d8484b9bc358.slice/crio-03185fb4cec856401bc475ae22150915eee9cf73c811087a66efb8812ddd3850 WatchSource:0}: Error finding container 03185fb4cec856401bc475ae22150915eee9cf73c811087a66efb8812ddd3850: Status 404 returned error can't find the container with id 03185fb4cec856401bc475ae22150915eee9cf73c811087a66efb8812ddd3850
Sep 11 18:12:14.375449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.375426    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-655bb94d9f-62jf9"]
Sep 11 18:12:14.378364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.378347    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.380882 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.380862    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 18:12:14.381151 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.381133    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 18:12:14.381242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.381155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ds27cp6a58spt\""
Sep 11 18:12:14.381242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.381180    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ds26r\""
Sep 11 18:12:14.381242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.381182    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 18:12:14.381417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.381136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 18:12:14.381903 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.381886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 18:12:14.397155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.397135    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-655bb94d9f-62jf9"]
Sep 11 18:12:14.400391 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.400312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9"]
Sep 11 18:12:14.402702 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:14.402670    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ef2858_8dbc_4a5f_97f7_fe8cc5702fab.slice/crio-5c61ba9d6d6f762de95c88492dfbda60dfc4c6b01c03e5ff64bb873d80a9ac0c WatchSource:0}: Error finding container 5c61ba9d6d6f762de95c88492dfbda60dfc4c6b01c03e5ff64bb873d80a9ac0c: Status 404 returned error can't find the container with id 5c61ba9d6d6f762de95c88492dfbda60dfc4c6b01c03e5ff64bb873d80a9ac0c
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469174    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-metrics-client-ca\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-tls\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469435    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-grpc-tls\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.472174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.469580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbgfn\" (UniqueName: \"kubernetes.io/projected/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-kube-api-access-wbgfn\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.474926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.474901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr" event={"ID":"a3c07754-c93c-4a26-a5c7-d8484b9bc358","Type":"ContainerStarted","Data":"04782994fbdf0ad6e9fcceedb6abc67ac303bbf5fbddfc494f9325941fbe9d6b"}
Sep 11 18:12:14.475038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.474990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr" event={"ID":"a3c07754-c93c-4a26-a5c7-d8484b9bc358","Type":"ContainerStarted","Data":"f3727a70df5c4a524fcff0b68affab50ed5e3016c87c62e9c1c0c468938644bc"}
Sep 11 18:12:14.475038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.475023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr" event={"ID":"a3c07754-c93c-4a26-a5c7-d8484b9bc358","Type":"ContainerStarted","Data":"03185fb4cec856401bc475ae22150915eee9cf73c811087a66efb8812ddd3850"}
Sep 11 18:12:14.476213 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.476189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9" event={"ID":"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab","Type":"ContainerStarted","Data":"5c61ba9d6d6f762de95c88492dfbda60dfc4c6b01c03e5ff64bb873d80a9ac0c"}
Sep 11 18:12:14.477928 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.477902    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ccc46eb-455f-4156-bab2-cd071622b409" containerID="df1c03f2e469e9b54c95578eb7892057ccc9c6f1b8f55ed69efe9ebb93082876" exitCode=0
Sep 11 18:12:14.478011 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.477947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmg88" event={"ID":"3ccc46eb-455f-4156-bab2-cd071622b409","Type":"ContainerDied","Data":"df1c03f2e469e9b54c95578eb7892057ccc9c6f1b8f55ed69efe9ebb93082876"}
Sep 11 18:12:14.570426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.570404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-grpc-tls\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.570527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.570437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbgfn\" (UniqueName: \"kubernetes.io/projected/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-kube-api-access-wbgfn\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.570527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.570468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.570527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.570485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-metrics-client-ca\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.570527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.570520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.570783 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.570577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-tls\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.571421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.571107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.571421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.571149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.571421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.571207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-metrics-client-ca\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.573247 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.573218    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-tls\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.573365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.573277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.573365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.573323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-grpc-tls\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.573475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.573374    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.573857 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.573840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.573900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.573872    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.579846 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.579820    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbgfn\" (UniqueName: \"kubernetes.io/projected/3a9bfd69-36cd-4858-8647-7b358c4f9a1c-kube-api-access-wbgfn\") pod \"thanos-querier-655bb94d9f-62jf9\" (UID: \"3a9bfd69-36cd-4858-8647-7b358c4f9a1c\") " pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.689713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.689684    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:14.964498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:14.964466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-655bb94d9f-62jf9"]
Sep 11 18:12:14.967142 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:14.967112    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9bfd69_36cd_4858_8647_7b358c4f9a1c.slice/crio-a74d7ceebcad90674ca19b3cb4df5e77110b9fd0dc1bf43ff392a56faf02e3b6 WatchSource:0}: Error finding container a74d7ceebcad90674ca19b3cb4df5e77110b9fd0dc1bf43ff392a56faf02e3b6: Status 404 returned error can't find the container with id a74d7ceebcad90674ca19b3cb4df5e77110b9fd0dc1bf43ff392a56faf02e3b6
Sep 11 18:12:15.256838 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.256802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:15.354388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.354326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:15.419459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.419427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:15.483427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.483389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"a74d7ceebcad90674ca19b3cb4df5e77110b9fd0dc1bf43ff392a56faf02e3b6"}
Sep 11 18:12:15.485914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.485886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmg88" event={"ID":"3ccc46eb-455f-4156-bab2-cd071622b409","Type":"ContainerStarted","Data":"bd0a8964e05e51700bb3724c174080b6bede099048ed4e5f7cfe6d494d193aaf"}
Sep 11 18:12:15.486044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.485933    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zmg88" event={"ID":"3ccc46eb-455f-4156-bab2-cd071622b409","Type":"ContainerStarted","Data":"80bf20293f2b0e19cdf70aabaf152cd53ca0384ee527f441bfeb4e558a3264d3"}
Sep 11 18:12:15.509983 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:15.509906    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zmg88" podStartSLOduration=2.0426084 podStartE2EDuration="3.50989214s" totalImagesPullingTime="1.400974942s" totalInitContainerRuntime="66.308798ms" isStatefulPod=true podCreationTimestamp="2026-09-11 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:15.509611232 +0000 UTC m=+104.102811790" watchObservedRunningTime="2026-09-11 18:12:15.50989214 +0000 UTC m=+104.103092687"
Sep 11 18:12:16.231506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.231423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:16.493283 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.493178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9" event={"ID":"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab","Type":"ContainerStarted","Data":"cd8734e08da2f2282e911073be06d485f6dda4c3f3d92876690b904308fb3f12"}
Sep 11 18:12:16.493283 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.493254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9" event={"ID":"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab","Type":"ContainerStarted","Data":"813eb3bfcbd7f1ae9dafd2f9ce9c732c65484e8d54c136a3422cabfa4d4dfeaa"}
Sep 11 18:12:16.493776 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.493302    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9" event={"ID":"24ef2858-8dbc-4a5f-97f7-fe8cc5702fab","Type":"ContainerStarted","Data":"e76a16e7e566da8bc725d0512edd8787aa100562491a7e096276890fd7a7f775"}
Sep 11 18:12:16.495713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.495686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr" event={"ID":"a3c07754-c93c-4a26-a5c7-d8484b9bc358","Type":"ContainerStarted","Data":"617854ff449b2b5727b7859fcc70c78efd201ce3dcbf8b96580c62b6acc7c952"}
Sep 11 18:12:16.518879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.518841    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-6xlf9" podStartSLOduration=2.971052415 podStartE2EDuration="4.518824872s" totalImagesPullingTime="1.547772457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:16.516556381 +0000 UTC m=+105.109756939" watchObservedRunningTime="2026-09-11 18:12:16.518824872 +0000 UTC m=+105.112025428"
Sep 11 18:12:16.539974 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:16.539930    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-n9wwr" podStartSLOduration=3.036320042 podStartE2EDuration="4.539911895s" totalImagesPullingTime="1.503591853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:16.539178505 +0000 UTC m=+105.132379062" watchObservedRunningTime="2026-09-11 18:12:16.539911895 +0000 UTC m=+105.133112453"
Sep 11 18:12:17.059742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.059698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:17.178582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.178495    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:17.273002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.272976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:17.501508 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.501471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"13cc97c93483e3fbff27d31ac5854ac8f2fd6cc417fe7ea62474f92958e36a3b"}
Sep 11 18:12:17.501836 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.501583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"58da42bc95cbb8781fdd72fd0800dcfacb126d7bf526e86e5cc4e52e31981d05"}
Sep 11 18:12:17.501836 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.501627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"f741b0ea2f8baa6b312f62d5c5052b0ecc8bc78427ba954b14f44526fda936cc"}
Sep 11 18:12:17.537254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.537225    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-66c5794976-gwqk2"]
Sep 11 18:12:17.540269 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.540242    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.543365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.543338    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 18:12:17.543572 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.543552    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 18:12:17.543797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.543782    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 18:12:17.544038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.544024    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 18:12:17.544297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.544281    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 18:12:17.548254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.548235    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dd4kl\""
Sep 11 18:12:17.553695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.553678    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 18:12:17.571255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.571207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66c5794976-gwqk2"]
Sep 11 18:12:17.698433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpw95\" (UniqueName: \"kubernetes.io/projected/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-kube-api-access-cpw95\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698576 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-secret-telemeter-client\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698576 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-metrics-client-ca\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698576 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-telemeter-client-tls\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698602    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-serving-certs-ca-bundle\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.698758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.698705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-federate-client-tls\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.799941 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.799915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-metrics-client-ca\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.799948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-telemeter-client-tls\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.799975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.799997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.800024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-serving-certs-ca-bundle\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800226 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.800091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-federate-client-tls\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800226 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.800135    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpw95\" (UniqueName: \"kubernetes.io/projected/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-kube-api-access-cpw95\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800226 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.800164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-secret-telemeter-client\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.800851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.800816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-metrics-client-ca\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.801054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.801032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.801054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.801037    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-serving-certs-ca-bundle\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.802646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.802621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.802786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.802768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-federate-client-tls\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.802954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.802936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-telemeter-client-tls\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.802997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.802952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-secret-telemeter-client\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.808920 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.808903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpw95\" (UniqueName: \"kubernetes.io/projected/58a9d0a8-0153-47bd-8437-b0e7d2506a4a-kube-api-access-cpw95\") pod \"telemeter-client-66c5794976-gwqk2\" (UID: \"58a9d0a8-0153-47bd-8437-b0e7d2506a4a\") " pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:17.848797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:17.848754    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2"
Sep 11 18:12:18.131755 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.131728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66c5794976-gwqk2"]
Sep 11 18:12:18.185064 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:18.185036    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a9d0a8_0153_47bd_8437_b0e7d2506a4a.slice/crio-7ace5c66f69dcbba32fef3be3e4a8b760328fd638398c0eb969039337e58e4fd WatchSource:0}: Error finding container 7ace5c66f69dcbba32fef3be3e4a8b760328fd638398c0eb969039337e58e4fd: Status 404 returned error can't find the container with id 7ace5c66f69dcbba32fef3be3e4a8b760328fd638398c0eb969039337e58e4fd
Sep 11 18:12:18.507772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.507739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"d87226da1ff5207344217787e9b7baf4d59f4c2b2fe9a452487f29607abd0d60"}
Sep 11 18:12:18.508132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.507817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"99b1a62facd022af02b4e61b19f44b4c380fbe8c45369f8c877784fd4bf7c801"}
Sep 11 18:12:18.508132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.507861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" event={"ID":"3a9bfd69-36cd-4858-8647-7b358c4f9a1c","Type":"ContainerStarted","Data":"d3061a92ddf3140bd1abf2f2230b410d794a89a7083d9c0a69a4f45fe1a542bb"}
Sep 11 18:12:18.508132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.507899    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:18.508944 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.508922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2" event={"ID":"58a9d0a8-0153-47bd-8437-b0e7d2506a4a","Type":"ContainerStarted","Data":"7ace5c66f69dcbba32fef3be3e4a8b760328fd638398c0eb969039337e58e4fd"}
Sep 11 18:12:18.530452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.530422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:18.539856 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.539819    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9" podStartSLOduration=1.456669811 podStartE2EDuration="4.539805821s" totalImagesPullingTime="3.08313601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:12:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:18.538555823 +0000 UTC m=+107.131756380" watchObservedRunningTime="2026-09-11 18:12:18.539805821 +0000 UTC m=+107.133006378"
Sep 11 18:12:18.543215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.543195    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:12:18.546985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.546970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.550110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.550087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 18:12:18.551330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.551310    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 18:12:18.551640 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.551623    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 18:12:18.551742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.551629    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 18:12:18.551800 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.551749    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 18:12:18.551944 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.551929    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 18:12:18.552031 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.551969    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 18:12:18.552087 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552033    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 18:12:18.552087 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 18:12:18.552179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552087    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 18:12:18.552179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552088    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-anga8nf42j0gl\""
Sep 11 18:12:18.552179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 18:12:18.552381 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552192    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 18:12:18.552381 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wjzr4\""
Sep 11 18:12:18.552740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.552723    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 18:12:18.555653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.555632    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 18:12:18.561868 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.561852    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 18:12:18.578073 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.578054    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:12:18.609584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.609564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:18.707543 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707543 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.707790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4slfj\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-kube-api-access-4slfj\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707936    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.707994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.708017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.708044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.708235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.708062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4slfj\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-kube-api-access-4slfj\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808757    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.808982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.808996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809479    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.809668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.809624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.811474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.811410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.811692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.812013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.812034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.812122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.812342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.812712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.812918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.813099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.813185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.813920 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.813895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.814001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.813953    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.814238 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.814220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.814342 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.814325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.814636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.814621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config-out\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.815054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.815035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-web-config\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.817693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.817670    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4slfj\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-kube-api-access-4slfj\") pod \"prometheus-k8s-0\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:18.858725 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:18.858696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:19.152377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:19.152346    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:12:19.154034 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:19.153992    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3fbc73b_f1bc_44d9_8a28_dbed643ca8c6.slice/crio-80a2b3d9351a9b3e069cb816c93f177c7d4498f4cb6fb3d69cd7259c0c497c4e WatchSource:0}: Error finding container 80a2b3d9351a9b3e069cb816c93f177c7d4498f4cb6fb3d69cd7259c0c497c4e: Status 404 returned error can't find the container with id 80a2b3d9351a9b3e069cb816c93f177c7d4498f4cb6fb3d69cd7259c0c497c4e
Sep 11 18:12:19.514101 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:19.514020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"80a2b3d9351a9b3e069cb816c93f177c7d4498f4cb6fb3d69cd7259c0c497c4e"}
Sep 11 18:12:20.520670 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:20.520627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2" event={"ID":"58a9d0a8-0153-47bd-8437-b0e7d2506a4a","Type":"ContainerStarted","Data":"c18645305ca7d27dc555efdb11c7d0bfc7708e8d33b8e10f67be5ed8e2a651aa"}
Sep 11 18:12:20.585767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:20.585747    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:12:21.009330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.009300    2563 patch_prober.go:28] interesting pod/image-registry-db649f896-wzjc8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 18:12:21.009484 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.009347    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-db649f896-wzjc8" podUID="37c05e10-e9f0-4f5b-beee-8617ab2ffc96" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:12:21.526745 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.526706    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2" event={"ID":"58a9d0a8-0153-47bd-8437-b0e7d2506a4a","Type":"ContainerStarted","Data":"4271352ff9a1d4b28cb646b24be1f56a8774819f5bbc4a486d744ec572966a86"}
Sep 11 18:12:21.527195 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.526784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2" event={"ID":"58a9d0a8-0153-47bd-8437-b0e7d2506a4a","Type":"ContainerStarted","Data":"53496cce87e2e623f2d2ff84b167fa307cf22598ba8317f64333eddb7e1ec552"}
Sep 11 18:12:21.528346 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.528325    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec" exitCode=0
Sep 11 18:12:21.528437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.528371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"}
Sep 11 18:12:21.560664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:21.560600    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-66c5794976-gwqk2" podStartSLOduration=2.276526675 podStartE2EDuration="4.560585021s" totalImagesPullingTime="2.284058346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:12:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:21.559247495 +0000 UTC m=+110.152448049" watchObservedRunningTime="2026-09-11 18:12:21.560585021 +0000 UTC m=+110.153785577"
Sep 11 18:12:22.213308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.213255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:22.317500 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.317471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:22.386377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.386351    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:22.425570 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.425548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db649f896-wzjc8"
Sep 11 18:12:22.466367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.466298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vk549"
Sep 11 18:12:22.613567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.613531    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:22.715081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.715047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:22.789378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:22.789348    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:24.522041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:24.522009    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-655bb94d9f-62jf9"
Sep 11 18:12:25.550687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.550650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"}
Sep 11 18:12:25.551084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.550733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"}
Sep 11 18:12:25.551084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.550783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"}
Sep 11 18:12:25.551084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.550825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"}
Sep 11 18:12:25.551084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.550863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"}
Sep 11 18:12:25.551084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.550902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerStarted","Data":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"}
Sep 11 18:12:25.599023 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.598988    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67f795677b-ck2df" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerName="registry" containerID="cri-o://c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6" gracePeriod=30
Sep 11 18:12:25.610740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.610701    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.950150422 podStartE2EDuration="7.610685675s" totalImagesPullingTime="4.63645127s" totalInitContainerRuntime="24.083983ms" isStatefulPod=false podCreationTimestamp="2026-09-11 18:12:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:25.608133827 +0000 UTC m=+114.201334396" watchObservedRunningTime="2026-09-11 18:12:25.610685675 +0000 UTC m=+114.203886232"
Sep 11 18:12:25.838036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.838015    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:12:25.975834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975806    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-certificates\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.975952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975841    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvjpx\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-kube-api-access-mvjpx\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.975952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a41f3cc4-74fe-419c-862c-8d120a60bf51-ca-trust-extracted\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.975952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-bound-sa-token\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.975952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975938    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-image-registry-private-configuration\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.975952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975953    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-installation-pull-secrets\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.976215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.975983    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.976215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.976110    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-trusted-ca\") pod \"a41f3cc4-74fe-419c-862c-8d120a60bf51\" (UID: \"a41f3cc4-74fe-419c-862c-8d120a60bf51\") "
Sep 11 18:12:25.976354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.976241    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:25.976620 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.976594    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-certificates\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:25.976704 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.976620    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:25.978384 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.978349    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-kube-api-access-mvjpx" (OuterVolumeSpecName: "kube-api-access-mvjpx") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "kube-api-access-mvjpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:25.978498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.978442    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:12:25.978595 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.978571    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:25.978648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.978619    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:12:25.978927 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.978903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:25.986979 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:25.986953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a41f3cc4-74fe-419c-862c-8d120a60bf51-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a41f3cc4-74fe-419c-862c-8d120a60bf51" (UID: "a41f3cc4-74fe-419c-862c-8d120a60bf51"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:26.077382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077328    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-bound-sa-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.077382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077349    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-image-registry-private-configuration\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.077382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077360    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a41f3cc4-74fe-419c-862c-8d120a60bf51-installation-pull-secrets\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.077382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077369    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-registry-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.077382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077378    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a41f3cc4-74fe-419c-862c-8d120a60bf51-trusted-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.077382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077385    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvjpx\" (UniqueName: \"kubernetes.io/projected/a41f3cc4-74fe-419c-862c-8d120a60bf51-kube-api-access-mvjpx\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.077615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.077394    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a41f3cc4-74fe-419c-862c-8d120a60bf51-ca-trust-extracted\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:12:26.555661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.555624    2563 generic.go:356] "Generic (PLEG): container finished" podID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerID="c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6" exitCode=0
Sep 11 18:12:26.556069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.555685    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f795677b-ck2df"
Sep 11 18:12:26.556069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.555713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f795677b-ck2df" event={"ID":"a41f3cc4-74fe-419c-862c-8d120a60bf51","Type":"ContainerDied","Data":"c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6"}
Sep 11 18:12:26.556069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.555778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f795677b-ck2df" event={"ID":"a41f3cc4-74fe-419c-862c-8d120a60bf51","Type":"ContainerDied","Data":"98491ac8f8389a101671b832e03747ae67caf3e3030c87b483005527901ab4e2"}
Sep 11 18:12:26.556069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.555803    2563 scope.go:117] "RemoveContainer" containerID="c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6"
Sep 11 18:12:26.565584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.565566    2563 scope.go:117] "RemoveContainer" containerID="c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6"
Sep 11 18:12:26.565874 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:12:26.565852    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6\": container with ID starting with c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6 not found: ID does not exist" containerID="c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6"
Sep 11 18:12:26.565924 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.565883    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6"} err="failed to get container status \"c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6\": rpc error: code = NotFound desc = could not find container \"c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6\": container with ID starting with c15e2bd8c3bf2d1760a01d70efcfc8d5bbaf7cb8413f6e3b0348ffb19a0edaa6 not found: ID does not exist"
Sep 11 18:12:26.585477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.585448    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f795677b-ck2df"]
Sep 11 18:12:26.590945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:26.590923    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67f795677b-ck2df"]
Sep 11 18:12:27.765810 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:27.765760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:27.850581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:27.850549    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:27.911402 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:27.911378    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" path="/var/lib/kubelet/pods/a41f3cc4-74fe-419c-862c-8d120a60bf51/volumes"
Sep 11 18:12:27.985030 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:27.985010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:28.080096 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:28.080075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:28.859214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:28.859187    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:12:30.191575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:30.191544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:30.298242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:30.298212    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:31.876581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:31.876551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:31.984139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:31.984113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:32.321274 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:32.321242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:32.424782 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:32.424754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:32.893441 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:32.893409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:32.997553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:32.997523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:33.109845 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:33.109815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:35.908499 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:35.908458    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:36.056584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:36.056551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:37.444294 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:37.444249    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:37.544094 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:37.544067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:40.974639 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:40.974601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.080827 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:41.080791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.698529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:41.698489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:12:41.700746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:41.700727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df55e017-4726-49f9-8507-41222acd67f0-metrics-certs\") pod \"network-metrics-daemon-qhmz2\" (UID: \"df55e017-4726-49f9-8507-41222acd67f0\") " pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:12:41.931134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:41.931109    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bvnsl\""
Sep 11 18:12:41.939021 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:41.939001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhmz2"
Sep 11 18:12:42.221738 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:42.221670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qhmz2"]
Sep 11 18:12:42.224200 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:12:42.224177    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf55e017_4726_49f9_8507_41222acd67f0.slice/crio-e4e03f4385aafb1ea6dc7046459766707a36138bb3231d6965737d0ddf63869e WatchSource:0}: Error finding container e4e03f4385aafb1ea6dc7046459766707a36138bb3231d6965737d0ddf63869e: Status 404 returned error can't find the container with id e4e03f4385aafb1ea6dc7046459766707a36138bb3231d6965737d0ddf63869e
Sep 11 18:12:42.286066 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:42.286040    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:42.360964 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:42.360939    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:42.618701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:42.618674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhmz2" event={"ID":"df55e017-4726-49f9-8507-41222acd67f0","Type":"ContainerStarted","Data":"e4e03f4385aafb1ea6dc7046459766707a36138bb3231d6965737d0ddf63869e"}
Sep 11 18:12:43.624977 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:43.624938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhmz2" event={"ID":"df55e017-4726-49f9-8507-41222acd67f0","Type":"ContainerStarted","Data":"e63c185c5e446a653d2c534504cb5fd5103ecaf910e3d11da41a4dd40125e3e2"}
Sep 11 18:12:43.625346 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:43.625008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhmz2" event={"ID":"df55e017-4726-49f9-8507-41222acd67f0","Type":"ContainerStarted","Data":"eecc47d9c045309c6a459e7e6bd9431d07af422d9615226d7c3ae615a51e9e1d"}
Sep 11 18:12:43.649228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:43.648811    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qhmz2" podStartSLOduration=130.768224277 podStartE2EDuration="2m11.648792557s" totalImagesPullingTime="880.56828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:10:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:12:43.648327526 +0000 UTC m=+132.241528084" watchObservedRunningTime="2026-09-11 18:12:43.648792557 +0000 UTC m=+132.241993116"
Sep 11 18:12:43.780107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:43.780075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:43.881778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:43.881714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:52.668736 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:52.668708    2563 generic.go:356] "Generic (PLEG): container finished" podID="f6687a09-2cda-40ca-8023-71593b98bf54" containerID="b89527e495e8ca069cde9d08204b1ca812233a02ff4a902a337fb340bf303cf4" exitCode=0
Sep 11 18:12:52.669026 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:52.668777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j" event={"ID":"f6687a09-2cda-40ca-8023-71593b98bf54","Type":"ContainerDied","Data":"b89527e495e8ca069cde9d08204b1ca812233a02ff4a902a337fb340bf303cf4"}
Sep 11 18:12:52.669069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:52.669042    2563 scope.go:117] "RemoveContainer" containerID="b89527e495e8ca069cde9d08204b1ca812233a02ff4a902a337fb340bf303cf4"
Sep 11 18:12:53.674692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.674659    2563 generic.go:356] "Generic (PLEG): container finished" podID="8035ce1c-4e18-4f71-bbd7-65643b813d0e" containerID="1831730a0ce65ed25b8efa7f351fa4b06eaba119d51c84b1542c46caf851aacc" exitCode=0
Sep 11 18:12:53.675111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.674744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jkm2r" event={"ID":"8035ce1c-4e18-4f71-bbd7-65643b813d0e","Type":"ContainerDied","Data":"1831730a0ce65ed25b8efa7f351fa4b06eaba119d51c84b1542c46caf851aacc"}
Sep 11 18:12:53.675170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.675152    2563 scope.go:117] "RemoveContainer" containerID="1831730a0ce65ed25b8efa7f351fa4b06eaba119d51c84b1542c46caf851aacc"
Sep 11 18:12:53.676964 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.676932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-mbn7j" event={"ID":"f6687a09-2cda-40ca-8023-71593b98bf54","Type":"ContainerStarted","Data":"a33f20e403052ed1e86163dbed6edd65d08f687669955de3884f7a62b69e8490"}
Sep 11 18:12:53.678627 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.678606    2563 generic.go:356] "Generic (PLEG): container finished" podID="7060fb14-474d-48f5-9e21-9067715c8951" containerID="6b5c5751fcf0f39773c2b90141c36739d5ded11fae4bf7b516c9bf38d61dbb88" exitCode=0
Sep 11 18:12:53.678711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.678661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf" event={"ID":"7060fb14-474d-48f5-9e21-9067715c8951","Type":"ContainerDied","Data":"6b5c5751fcf0f39773c2b90141c36739d5ded11fae4bf7b516c9bf38d61dbb88"}
Sep 11 18:12:53.678910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:53.678897    2563 scope.go:117] "RemoveContainer" containerID="6b5c5751fcf0f39773c2b90141c36739d5ded11fae4bf7b516c9bf38d61dbb88"
Sep 11 18:12:54.684255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:54.684219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-pxfqf" event={"ID":"7060fb14-474d-48f5-9e21-9067715c8951","Type":"ContainerStarted","Data":"4576c2851dea23ceb7157ad7dc84ab7f5a3be0a38c54d44f68c28f2c5b4d9245"}
Sep 11 18:12:54.685951 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:54.685927    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-jkm2r" event={"ID":"8035ce1c-4e18-4f71-bbd7-65643b813d0e","Type":"ContainerStarted","Data":"8a55ac3026a0d5b336723f2285e3061ad9c379fd4122f9999b4900443c6b62ae"}
Sep 11 18:12:55.966702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:55.966667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:56.044954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:12:56.044923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:05.841598 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:05.841563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:05.936462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:05.936428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:18.859658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:18.859623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:18.879713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:18.879682    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:19.814190 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:19.814162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:28.455050 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:28.454967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:28.541974 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:28.541948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:37.221885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.221850    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:13:37.222335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.222134    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="prometheus" containerID="cri-o://4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb" gracePeriod=600
Sep 11 18:13:37.222335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.222173    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy" containerID="cri-o://061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b" gracePeriod=600
Sep 11 18:13:37.222335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.222196    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="thanos-sidecar" containerID="cri-o://6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba" gracePeriod=600
Sep 11 18:13:37.222335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.222212    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-web" containerID="cri-o://41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728" gracePeriod=600
Sep 11 18:13:37.222335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.222236    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="config-reloader" containerID="cri-o://2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580" gracePeriod=600
Sep 11 18:13:37.222335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.222250    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d" gracePeriod=600
Sep 11 18:13:37.462558 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.462536    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:37.616972 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.616947    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.616981    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-metrics-client-certs\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.616999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-grpc-tls\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-tls-assets\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-web-config\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617104    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-serving-certs-ca-bundle\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-kubelet-serving-ca-bundle\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-2\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617184    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-trusted-ca-bundle\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617211    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4slfj\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-kube-api-access-4slfj\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617248    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-1\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617313    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-db\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617341    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617389    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-thanos-prometheus-http-client-file\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617420    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-kube-rbac-proxy\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617463    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-metrics-client-ca\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.617491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-0\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.618024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617515    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.618024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617550    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config-out\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.618024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617592    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-tls\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.618024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\" (UID: \"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6\") "
Sep 11 18:13:37.618024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617877    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.618024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.617892    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.618350 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.618177    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.619646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.619610    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.619827 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.619773    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.619917 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.619846    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.619982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.619963    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.620364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.620460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620426    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:13:37.620526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.620585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620553    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.620641 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-kube-api-access-4slfj" (OuterVolumeSpecName: "kube-api-access-4slfj") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "kube-api-access-4slfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:13:37.620868 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.620961 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.620910    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:37.621925 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.621900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:37.622351 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.622286    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config" (OuterVolumeSpecName: "config") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.622631 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.622610    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.622727 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.622711    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.622802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.622787    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config-out" (OuterVolumeSpecName: "config-out") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:37.630906 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.630882    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-web-config" (OuterVolumeSpecName: "web-config") pod "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" (UID: "f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:13:37.719000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.718971    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.718998    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719013    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-metrics-client-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719026    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-grpc-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719039    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-tls-assets\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719050    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-web-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719062    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719076    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719089    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719102    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4slfj\" (UniqueName: \"kubernetes.io/projected/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-kube-api-access-4slfj\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719116    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719130    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-db\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719144    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719157    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719172    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-kube-rbac-proxy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719186    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-configmap-metrics-client-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719199    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719213    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-config-out\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.719561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.719226    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:13:37.870554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870493    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d" exitCode=0
Sep 11 18:13:37.870554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870512    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b" exitCode=0
Sep 11 18:13:37.870554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870519    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728" exitCode=0
Sep 11 18:13:37.870554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870525    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba" exitCode=0
Sep 11 18:13:37.870554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870530    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580" exitCode=0
Sep 11 18:13:37.870554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870535    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb" exitCode=0
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"}
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870591    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"}
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"}
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"}
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870732    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"}
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"}
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870707    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.870901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.870789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6","Type":"ContainerDied","Data":"80a2b3d9351a9b3e069cb816c93f177c7d4498f4cb6fb3d69cd7259c0c497c4e"}
Sep 11 18:13:37.879427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.879412    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.885735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.885717    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.892095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.892077    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.894531 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.894510    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:13:37.899066 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.899046    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:13:37.899932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.899913    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.906555 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.906539    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.911599 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.911576    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" path="/var/lib/kubelet/pods/f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6/volumes"
Sep 11 18:13:37.914011 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.913994    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.920603 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.920586    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.920856 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.920838    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.920902 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.920864    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"} err="failed to get container status \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist"
Sep 11 18:13:37.920902 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.920879    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.921140 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.921125    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.921193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921143    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"} err="failed to get container status \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist"
Sep 11 18:13:37.921193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921156    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.921417 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.921399    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.921461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921423    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"} err="failed to get container status \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist"
Sep 11 18:13:37.921461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921439    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.921682 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.921663    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.921730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921687    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"} err="failed to get container status \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist"
Sep 11 18:13:37.921730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921701    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.921928 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.921913    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.921962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921931    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"} err="failed to get container status \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist"
Sep 11 18:13:37.921962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.921945    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.922164 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.922149    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.922205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922169    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"} err="failed to get container status \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist"
Sep 11 18:13:37.922205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922183    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.922439 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:13:37.922424    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.922486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922441    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"} err="failed to get container status \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist"
Sep 11 18:13:37.922486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922453    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.922684 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922667    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"} err="failed to get container status \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist"
Sep 11 18:13:37.922684 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922683    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.922911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922893    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"} err="failed to get container status \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist"
Sep 11 18:13:37.922911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.922910    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.923143 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923126    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"} err="failed to get container status \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist"
Sep 11 18:13:37.923181 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923143    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.923357 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923340    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"} err="failed to get container status \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist"
Sep 11 18:13:37.923420 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923357    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.923544 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923527    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"} err="failed to get container status \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist"
Sep 11 18:13:37.923586 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923544    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.923740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923725    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"} err="failed to get container status \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist"
Sep 11 18:13:37.923781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923741    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.923958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923940    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"} err="failed to get container status \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist"
Sep 11 18:13:37.923958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.923957    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.924148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924133    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"} err="failed to get container status \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist"
Sep 11 18:13:37.924186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924148    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.924358 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924342    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"} err="failed to get container status \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist"
Sep 11 18:13:37.924408 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924357    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.924550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924534    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"} err="failed to get container status \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist"
Sep 11 18:13:37.924588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924550    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.924734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924717    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"} err="failed to get container status \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist"
Sep 11 18:13:37.924775 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924734    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.924939 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924923    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"} err="failed to get container status \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist"
Sep 11 18:13:37.924939 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.924937    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.925121 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925106    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"} err="failed to get container status \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist"
Sep 11 18:13:37.925158 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925121    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.925317 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925301    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"} err="failed to get container status \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist"
Sep 11 18:13:37.925369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925317    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.925508 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925491    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"} err="failed to get container status \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist"
Sep 11 18:13:37.925550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925508    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.925722 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925707    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"} err="failed to get container status \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist"
Sep 11 18:13:37.925766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925723    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.925949 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925931    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"} err="failed to get container status \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist"
Sep 11 18:13:37.926005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.925958    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.926339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.926302    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"} err="failed to get container status \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist"
Sep 11 18:13:37.926339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.926333    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.926643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.926622    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"} err="failed to get container status \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist"
Sep 11 18:13:37.926706 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.926644    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.926914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.926896    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"} err="failed to get container status \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist"
Sep 11 18:13:37.926967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.926916    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.927171 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927153    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"} err="failed to get container status \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist"
Sep 11 18:13:37.927237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927172    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.927413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927396    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"} err="failed to get container status \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist"
Sep 11 18:13:37.927463 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927414    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.927689 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927672    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"} err="failed to get container status \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist"
Sep 11 18:13:37.927689 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927687    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.927928 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927907    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"} err="failed to get container status \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist"
Sep 11 18:13:37.928000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.927929    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.928203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928173    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"} err="failed to get container status \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist"
Sep 11 18:13:37.928203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928200    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.928438 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928420    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"} err="failed to get container status \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist"
Sep 11 18:13:37.928513 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928441    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.928678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928659    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"} err="failed to get container status \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist"
Sep 11 18:13:37.928744 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928679    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.928926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928905    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"} err="failed to get container status \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist"
Sep 11 18:13:37.928970 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.928927    2563 scope.go:117] "RemoveContainer" containerID="1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"
Sep 11 18:13:37.929128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929111    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d"} err="failed to get container status \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": rpc error: code = NotFound desc = could not find container \"1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d\": container with ID starting with 1390293be8775c27666034a06849203a3c566d9f4fa3c77f96f342f2c061a64d not found: ID does not exist"
Sep 11 18:13:37.929171 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929129    2563 scope.go:117] "RemoveContainer" containerID="061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"
Sep 11 18:13:37.929380 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929362    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b"} err="failed to get container status \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": rpc error: code = NotFound desc = could not find container \"061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b\": container with ID starting with 061f7255ada7bb475535813fcd9b51a28c44bad7012d26702cf2d8d09ac4ae5b not found: ID does not exist"
Sep 11 18:13:37.929458 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929382    2563 scope.go:117] "RemoveContainer" containerID="41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"
Sep 11 18:13:37.929614 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929598    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728"} err="failed to get container status \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": rpc error: code = NotFound desc = could not find container \"41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728\": container with ID starting with 41547a5017fa7ba71e15a5455ccf64d88ca99792ae6a0d42edd42bf30288c728 not found: ID does not exist"
Sep 11 18:13:37.929666 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929615    2563 scope.go:117] "RemoveContainer" containerID="6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"
Sep 11 18:13:37.929809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929792    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba"} err="failed to get container status \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": rpc error: code = NotFound desc = could not find container \"6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba\": container with ID starting with 6f3de17256d6496f3fd82f81c39452279602bd080036de79547ad2e15c6f71ba not found: ID does not exist"
Sep 11 18:13:37.929858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.929810    2563 scope.go:117] "RemoveContainer" containerID="2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"
Sep 11 18:13:37.930032 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.930014    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580"} err="failed to get container status \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": rpc error: code = NotFound desc = could not find container \"2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580\": container with ID starting with 2678555eed616188166d046dbdc3a64dd5af3cc20cd82df98161ac32965da580 not found: ID does not exist"
Sep 11 18:13:37.930098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.930033    2563 scope.go:117] "RemoveContainer" containerID="4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"
Sep 11 18:13:37.930252 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.930235    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb"} err="failed to get container status \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": rpc error: code = NotFound desc = could not find container \"4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb\": container with ID starting with 4729cbf1963476ea44ae0340989c6d4d4b2cf9bcc68e5b29b51b524f65311bbb not found: ID does not exist"
Sep 11 18:13:37.930321 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.930253    2563 scope.go:117] "RemoveContainer" containerID="a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"
Sep 11 18:13:37.930484 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.930467    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec"} err="failed to get container status \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": rpc error: code = NotFound desc = could not find container \"a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec\": container with ID starting with a248d22e420229f4ce56960497d2f0986f61169f4909f32912755bffec4c5bec not found: ID does not exist"
Sep 11 18:13:37.936002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.935984    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:13:37.936325 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936310    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="init-config-reloader"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936328    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="init-config-reloader"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936338    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936346    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936361    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-thanos"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936370    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-thanos"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936386    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="prometheus"
Sep 11 18:13:37.936407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936394    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="prometheus"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936412    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="config-reloader"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936420    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="config-reloader"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936432    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="thanos-sidecar"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936441    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="thanos-sidecar"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936452    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerName="registry"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936461    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerName="registry"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936475    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-web"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936485    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-web"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936572    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936586    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a41f3cc4-74fe-419c-862c-8d120a60bf51" containerName="registry"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936597    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-thanos"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936608    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="kube-rbac-proxy-web"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936616    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="thanos-sidecar"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936624    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="prometheus"
Sep 11 18:13:37.936749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.936632    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3fbc73b-f1bc-44d9-8a28-dbed643ca8c6" containerName="config-reloader"
Sep 11 18:13:37.942062 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.942045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:37.946567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.946550    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-wjzr4\""
Sep 11 18:13:37.946658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.946551    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 18:13:37.946729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.946679    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 18:13:37.947046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947001    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 18:13:37.947046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 18:13:37.947046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947027    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 18:13:37.947046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947040    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 18:13:37.947625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947603    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-anga8nf42j0gl\""
Sep 11 18:13:37.947738 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947665    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 18:13:37.947738 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947717    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 18:13:37.947885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947744    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 18:13:37.947885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947603    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 18:13:37.947986 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.947944    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 18:13:37.948202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.948181    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 18:13:37.948303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.948285    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 18:13:37.950437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.950417    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 18:13:37.958777 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.955706    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 18:13:37.966340 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:37.966318    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:13:38.122025 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.121974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122025 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.121999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122025 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa650c97-9230-4c35-a6d8-e81a3975ade2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whbxx\" (UniqueName: \"kubernetes.io/projected/fa650c97-9230-4c35-a6d8-e81a3975ade2-kube-api-access-whbxx\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa650c97-9230-4c35-a6d8-e81a3975ade2-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122567    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-config\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.122618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.122580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223409 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa650c97-9230-4c35-a6d8-e81a3975ade2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whbxx\" (UniqueName: \"kubernetes.io/projected/fa650c97-9230-4c35-a6d8-e81a3975ade2-kube-api-access-whbxx\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.223729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223679    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224045 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223767    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224045 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.223712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa650c97-9230-4c35-a6d8-e81a3975ade2-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224145 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224145 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224145 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-config\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224214    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.224313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.224830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa650c97-9230-4c35-a6d8-e81a3975ade2-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226300    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226543    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-config\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.226950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227207    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.228703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.227982    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa650c97-9230-4c35-a6d8-e81a3975ade2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.229281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.228427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.229281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.228999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa650c97-9230-4c35-a6d8-e81a3975ade2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.229281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.229065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.229281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.229073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.229596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.229577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa650c97-9230-4c35-a6d8-e81a3975ade2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.232425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.232402    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whbxx\" (UniqueName: \"kubernetes.io/projected/fa650c97-9230-4c35-a6d8-e81a3975ade2-kube-api-access-whbxx\") pod \"prometheus-k8s-0\" (UID: \"fa650c97-9230-4c35-a6d8-e81a3975ade2\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.251386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.251365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:38.539112 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.538449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 18:13:38.541102 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:13:38.541076    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa650c97_9230_4c35_a6d8_e81a3975ade2.slice/crio-bd514a17ea011ca4a5f85a28916be09d752b30a7f8f77043a970b5e9bc187126 WatchSource:0}: Error finding container bd514a17ea011ca4a5f85a28916be09d752b30a7f8f77043a970b5e9bc187126: Status 404 returned error can't find the container with id bd514a17ea011ca4a5f85a28916be09d752b30a7f8f77043a970b5e9bc187126
Sep 11 18:13:38.876907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.876846    2563 generic.go:356] "Generic (PLEG): container finished" podID="fa650c97-9230-4c35-a6d8-e81a3975ade2" containerID="73f280f69c8b1935625515208978906b2672e597805d2e637143364aca984af8" exitCode=0
Sep 11 18:13:38.877036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.876940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerDied","Data":"73f280f69c8b1935625515208978906b2672e597805d2e637143364aca984af8"}
Sep 11 18:13:38.877036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:38.877025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"bd514a17ea011ca4a5f85a28916be09d752b30a7f8f77043a970b5e9bc187126"}
Sep 11 18:13:39.890511 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.890476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"a29e3bcc40c44e0228b150f88e7e7d9d3fe38bd54c768bee75dc7cb0fba12b82"}
Sep 11 18:13:39.890872 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.890562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"a6bb6bc70373b537ccc83439aea002f59c63b814e6f402aacd9a03d5c156e926"}
Sep 11 18:13:39.890872 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.890591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"f10ca93a5f41ea5cf36f96e0dbe762060b7ac44105533345dbf9a808a081ab3e"}
Sep 11 18:13:39.890872 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.890617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"27a960295d454b084725a4953443c3c09e651ef797e7817d823cda4940066ce0"}
Sep 11 18:13:39.890872 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.890639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"5a011cb76abc8182288e1adce22eb1eec5b500614240679aff9642f14197fbf5"}
Sep 11 18:13:39.890872 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.890662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa650c97-9230-4c35-a6d8-e81a3975ade2","Type":"ContainerStarted","Data":"8206d68f00056c87cdc6d222bacb534f64aa103494e785ac34d971b195467c77"}
Sep 11 18:13:39.927617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:39.927270    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.906141948 podStartE2EDuration="2.927237594s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.095646ms" isStatefulPod=false podCreationTimestamp="2026-09-11 18:13:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:13:39.926285817 +0000 UTC m=+188.519486375" watchObservedRunningTime="2026-09-11 18:13:39.927237594 +0000 UTC m=+188.520438152"
Sep 11 18:13:40.367714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:40.367686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:40.448093 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:40.448061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:43.252059 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:43.252023    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:13:46.463156 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:46.463117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:46.604159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:46.604132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:46.704771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:46.704748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:47.148799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:47.148763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:47.250703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:47.250668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:55.986609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:55.986576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:56.083287 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:56.083242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:57.187086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:57.187055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:57.291907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:57.291880    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:57.390869 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:13:57.390841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:24.446432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:24.446395    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:24.520695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:24.520669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:34.607886 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:34.607852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:34.722993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:34.722955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:38.252427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:38.252396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:14:38.267177 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:38.267156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:14:39.144963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:14:39.144935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 18:15:02.500638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:02.500555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:03.938878 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:03.938847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:06.311111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:06.311077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:06.414969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:06.414934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:07.722490 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:07.722453    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:07.885564 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:07.885534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:08.103083 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:08.103054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:08.205159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:08.205138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:10.452792 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:10.452764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:10.740240 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:10.740155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:11.595795 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:11.595760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:13.083365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:13.083331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:13.162783 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:13.162752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:15.953604 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:15.953566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:16.071367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:16.071334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:18.093695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:18.093659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:19.169329 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:19.169299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:19.272062 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:19.272032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:26.872867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:26.872834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:26.973316 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:26.973256    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:29.582430 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:29.582393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:31.808179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:31.808154    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:15:31.810245 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:31.810225    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:15:31.811390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:31.811371    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 18:15:40.324411 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:40.324376    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:40.426872 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:40.426843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:41.243095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.243058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:41.696212 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.696181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:41.799691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.799656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:41.880542 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.880519    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"]
Sep 11 18:15:41.883594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.883578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:41.886035 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.886013    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 18:15:41.886150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.886055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 18:15:41.886215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.886151    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 18:15:41.886354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.886323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 18:15:41.886426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.886388    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m5jv4\""
Sep 11 18:15:41.902446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.902423    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"]
Sep 11 18:15:41.925477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.925451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvds5\" (UniqueName: \"kubernetes.io/projected/b4972f6f-57c6-4d5d-957a-753453be6f05-kube-api-access-tvds5\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:41.925608 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.925520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4972f6f-57c6-4d5d-957a-753453be6f05-manifests\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:41.925608 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.925560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4972f6f-57c6-4d5d-957a-753453be6f05-tmp\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:41.925608 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.925578    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4972f6f-57c6-4d5d-957a-753453be6f05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:41.925752 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:41.925695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4972f6f-57c6-4d5d-957a-753453be6f05-webhook-cert\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.027066 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.027045    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4972f6f-57c6-4d5d-957a-753453be6f05-webhook-cert\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.027163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.027090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvds5\" (UniqueName: \"kubernetes.io/projected/b4972f6f-57c6-4d5d-957a-753453be6f05-kube-api-access-tvds5\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.027163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.027140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4972f6f-57c6-4d5d-957a-753453be6f05-manifests\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.027240 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.027193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4972f6f-57c6-4d5d-957a-753453be6f05-tmp\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.027240 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.027221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4972f6f-57c6-4d5d-957a-753453be6f05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.027546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.027531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4972f6f-57c6-4d5d-957a-753453be6f05-manifests\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.029603 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.029579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4972f6f-57c6-4d5d-957a-753453be6f05-tmp\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.029683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.029660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4972f6f-57c6-4d5d-957a-753453be6f05-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.029722 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.029701    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4972f6f-57c6-4d5d-957a-753453be6f05-webhook-cert\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.040126 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.040100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvds5\" (UniqueName: \"kubernetes.io/projected/b4972f6f-57c6-4d5d-957a-753453be6f05-kube-api-access-tvds5\") pod \"opendatahub-operator-controller-manager-657fcdcb9c-nrzn2\" (UID: \"b4972f6f-57c6-4d5d-957a-753453be6f05\") " pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.047675 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.047655    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:42.140479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.140448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:42.194440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.194420    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:42.257425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.257398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:42.353565 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.353494    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:42.458179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.458150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:42.476042 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.476024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"]
Sep 11 18:15:42.478859 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:15:42.478836    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4972f6f_57c6_4d5d_957a_753453be6f05.slice/crio-9b516e2e2b26f24d21323214a9bef01557e729683398ce0d33956d8abf2e610d WatchSource:0}: Error finding container 9b516e2e2b26f24d21323214a9bef01557e729683398ce0d33956d8abf2e610d: Status 404 returned error can't find the container with id 9b516e2e2b26f24d21323214a9bef01557e729683398ce0d33956d8abf2e610d
Sep 11 18:15:42.480527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.480511    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:15:42.559774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.559749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:42.659998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:42.659943    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:43.377083 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:43.377046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2" event={"ID":"b4972f6f-57c6-4d5d-957a-753453be6f05","Type":"ContainerStarted","Data":"9b516e2e2b26f24d21323214a9bef01557e729683398ce0d33956d8abf2e610d"}
Sep 11 18:15:46.392552 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:46.392513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2" event={"ID":"b4972f6f-57c6-4d5d-957a-753453be6f05","Type":"ContainerStarted","Data":"8a25e6cad2b64011e1825489deeabd52da54740e855738eaa9c89eff21dc544e"}
Sep 11 18:15:47.399099 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:47.399067    2563 generic.go:356] "Generic (PLEG): container finished" podID="b4972f6f-57c6-4d5d-957a-753453be6f05" containerID="8a25e6cad2b64011e1825489deeabd52da54740e855738eaa9c89eff21dc544e" exitCode=0
Sep 11 18:15:47.399539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:47.399150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2" event={"ID":"b4972f6f-57c6-4d5d-957a-753453be6f05","Type":"ContainerDied","Data":"8a25e6cad2b64011e1825489deeabd52da54740e855738eaa9c89eff21dc544e"}
Sep 11 18:15:48.405114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:48.405078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2" event={"ID":"b4972f6f-57c6-4d5d-957a-753453be6f05","Type":"ContainerStarted","Data":"990222c724df21e9b7f73d0aa53e23b94d4dd44195173067fa296bfd82b126ea"}
Sep 11 18:15:48.405558 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:48.405146    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:15:48.436632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:48.436587    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2" podStartSLOduration=2.89354206 podStartE2EDuration="7.436571006s" totalImagesPullingTime="3.461985427s" totalInitContainerRuntime="1.081043519s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:15:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:48.43578549 +0000 UTC m=+317.028986047" watchObservedRunningTime="2026-09-11 18:15:48.436571006 +0000 UTC m=+317.029771588"
Sep 11 18:15:53.861349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:53.861296    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:55.261209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:55.261173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:59.412635 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:15:59.412603    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-657fcdcb9c-nrzn2"
Sep 11 18:16:01.755376 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:01.755338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:01.971885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:01.971850    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:02.362746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.362706    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:02.399888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.399858    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"]
Sep 11 18:16:02.406017 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.405997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.409405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.409381    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 18:16:02.409609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.409585    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 18:16:02.409609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.409596    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 18:16:02.409751 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.409639    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-smlfd\""
Sep 11 18:16:02.410096 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.410075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 18:16:02.418790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.418769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"]
Sep 11 18:16:02.479940 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.479909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/465c25cf-1958-4d11-9ab4-d6728ebcd361-tmp\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.480038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.479993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9xnv\" (UniqueName: \"kubernetes.io/projected/465c25cf-1958-4d11-9ab4-d6728ebcd361-kube-api-access-p9xnv\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.480038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.480031    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465c25cf-1958-4d11-9ab4-d6728ebcd361-tls-certs\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.580958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.580932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/465c25cf-1958-4d11-9ab4-d6728ebcd361-tmp\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.581063 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.580993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9xnv\" (UniqueName: \"kubernetes.io/projected/465c25cf-1958-4d11-9ab4-d6728ebcd361-kube-api-access-p9xnv\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.581063 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.581037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465c25cf-1958-4d11-9ab4-d6728ebcd361-tls-certs\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.583230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.583211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/465c25cf-1958-4d11-9ab4-d6728ebcd361-tmp\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.583473 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.583454    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465c25cf-1958-4d11-9ab4-d6728ebcd361-tls-certs\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.600123 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.600105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9xnv\" (UniqueName: \"kubernetes.io/projected/465c25cf-1958-4d11-9ab4-d6728ebcd361-kube-api-access-p9xnv\") pod \"kube-auth-proxy-b485767cb-wgfwb\" (UID: \"465c25cf-1958-4d11-9ab4-d6728ebcd361\") " pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.703147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.703086    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"]
Sep 11 18:16:02.705443 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.705426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.710865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.710845    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 18:16:02.711144 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.711127    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 18:16:02.711452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.711435    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 18:16:02.711560 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.711540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7c54n\""
Sep 11 18:16:02.711841 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.711811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 18:16:02.715977 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.715960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"
Sep 11 18:16:02.716050 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.716001    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 18:16:02.720787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.720743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:02.725560 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.725539    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"]
Sep 11 18:16:02.782988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.782960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-cert\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.783485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.783461    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77k7r\" (UniqueName: \"kubernetes.io/projected/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-kube-api-access-77k7r\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.783646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.783626    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-manager-config\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.783801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.783784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-metrics-cert\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.884559 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.884534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77k7r\" (UniqueName: \"kubernetes.io/projected/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-kube-api-access-77k7r\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.884651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.884570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-manager-config\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.884651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.884598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-metrics-cert\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.884651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.884632    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-cert\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.885356 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.885335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-manager-config\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.886913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.886888    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-cert\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.886986 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.886925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-metrics-cert\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:02.924754 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:02.924729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77k7r\" (UniqueName: \"kubernetes.io/projected/253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8-kube-api-access-77k7r\") pod \"lws-controller-manager-6dd5f9dbb5-m69w6\" (UID: \"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8\") " pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:03.013315 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:16:03.013273    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod465c25cf_1958_4d11_9ab4_d6728ebcd361.slice/crio-ece5f69386ae28b92acd1224cc55091444a2f2c77585bd687fc3f146a1e6f30b WatchSource:0}: Error finding container ece5f69386ae28b92acd1224cc55091444a2f2c77585bd687fc3f146a1e6f30b: Status 404 returned error can't find the container with id ece5f69386ae28b92acd1224cc55091444a2f2c77585bd687fc3f146a1e6f30b
Sep 11 18:16:03.014309 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:03.014292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:03.015473 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:03.015454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b485767cb-wgfwb"]
Sep 11 18:16:03.322334 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:03.322225    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"]
Sep 11 18:16:03.324550 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:16:03.324526    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253e6e7d_d83c_4d5e_9c24_9fc1e8aa28a8.slice/crio-889a466902824bb4f436f7fb6f61453cb65df1e8688e30a2e2cf40893fa401fe WatchSource:0}: Error finding container 889a466902824bb4f436f7fb6f61453cb65df1e8688e30a2e2cf40893fa401fe: Status 404 returned error can't find the container with id 889a466902824bb4f436f7fb6f61453cb65df1e8688e30a2e2cf40893fa401fe
Sep 11 18:16:03.473678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:03.473654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6" event={"ID":"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8","Type":"ContainerStarted","Data":"889a466902824bb4f436f7fb6f61453cb65df1e8688e30a2e2cf40893fa401fe"}
Sep 11 18:16:03.474883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:03.474858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb" event={"ID":"465c25cf-1958-4d11-9ab4-d6728ebcd361","Type":"ContainerStarted","Data":"ece5f69386ae28b92acd1224cc55091444a2f2c77585bd687fc3f146a1e6f30b"}
Sep 11 18:16:04.747635 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:04.747600    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:04.857308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:04.857277    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:06.236870 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:06.236841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:06.493594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:06.493517    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb" event={"ID":"465c25cf-1958-4d11-9ab4-d6728ebcd361","Type":"ContainerStarted","Data":"78dcfe0cabcbb9421de399f56bbf72f4d791def1fe869b73275eb5330a118ed0"}
Sep 11 18:16:06.513343 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:06.513296    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b485767cb-wgfwb" podStartSLOduration=1.318038479 podStartE2EDuration="4.513281941s" totalImagesPullingTime="3.195243462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:06.512233283 +0000 UTC m=+335.105433837" watchObservedRunningTime="2026-09-11 18:16:06.513281941 +0000 UTC m=+335.106482488"
Sep 11 18:16:10.566588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:10.566557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:10.676824 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:10.676779    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:11.531214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:11.531185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:11.845157 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:11.845090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:11.937337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:11.937308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:12.746039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:12.745987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:12.845825 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:12.845791    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:13.019453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:13.019383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:13.109647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:13.109611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:13.377520 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:13.377480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:13.807315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:13.807284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:14.534581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:14.534545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6" event={"ID":"253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8","Type":"ContainerStarted","Data":"d290b18fa319e0452c923ce5a9e0404535a6936e070bd097927fd3cd30c69877"}
Sep 11 18:16:14.534946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:14.534624    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:16:14.561069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:14.561022    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6" podStartSLOduration=1.8072640020000001 podStartE2EDuration="12.561003484s" totalImagesPullingTime="10.753739482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:16:14.55875663 +0000 UTC m=+343.151957186" watchObservedRunningTime="2026-09-11 18:16:14.561003484 +0000 UTC m=+343.154204043"
Sep 11 18:16:23.924691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:23.924588    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:24.069464 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:24.069425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:24.255061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:24.254946    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:24.699563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:24.699534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:25.320625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:25.320574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:25.512255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:25.512210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:16:25.541585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:16:25.541559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6dd5f9dbb5-m69w6"
Sep 11 18:17:13.760304 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:13.760245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:13.894189 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:13.894156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:38.251915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:38.251877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:40.595825 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:40.595793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:40.789871 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:40.789839    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:40.888192 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:40.888112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:41.071515 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:41.071484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:43.279195 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:43.279155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:43.455860 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:43.455821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:44.621274 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:44.621237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:45.602719 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:45.602683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:45.934278 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:45.934179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:46.065850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:46.065818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:46.601724 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:46.601670    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:46.928465 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:46.928384    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:47.076363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:47.076334    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:49.880401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.880368    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"]
Sep 11 18:17:49.883879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.883859    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:49.889396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.889372    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 18:17:49.889525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.889383    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 18:17:49.890090 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.890066    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zwkz4\""
Sep 11 18:17:49.912414 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.912385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"]
Sep 11 18:17:49.983914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.983878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d01ea7b-c17c-4931-8f48-a21e74155ed1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:49.984099 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:49.983993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hts6\" (UniqueName: \"kubernetes.io/projected/2d01ea7b-c17c-4931-8f48-a21e74155ed1-kube-api-access-7hts6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:50.085207 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.085154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hts6\" (UniqueName: \"kubernetes.io/projected/2d01ea7b-c17c-4931-8f48-a21e74155ed1-kube-api-access-7hts6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:50.085394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.085241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d01ea7b-c17c-4931-8f48-a21e74155ed1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:50.085623 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.085602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d01ea7b-c17c-4931-8f48-a21e74155ed1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:50.098103 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.098078    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hts6\" (UniqueName: \"kubernetes.io/projected/2d01ea7b-c17c-4931-8f48-a21e74155ed1-kube-api-access-7hts6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:50.194242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.194142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:50.472123 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.472098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"]
Sep 11 18:17:50.473930 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:17:50.473901    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d01ea7b_c17c_4931_8f48_a21e74155ed1.slice/crio-1d48ddfe63bebe48aa58399e0bb324e574b7f89689d768f911cd4778f5564e44 WatchSource:0}: Error finding container 1d48ddfe63bebe48aa58399e0bb324e574b7f89689d768f911cd4778f5564e44: Status 404 returned error can't find the container with id 1d48ddfe63bebe48aa58399e0bb324e574b7f89689d768f911cd4778f5564e44
Sep 11 18:17:50.869339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.869304    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:50.980492 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.980456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6" event={"ID":"2d01ea7b-c17c-4931-8f48-a21e74155ed1","Type":"ContainerStarted","Data":"1d48ddfe63bebe48aa58399e0bb324e574b7f89689d768f911cd4778f5564e44"}
Sep 11 18:17:50.998859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:50.998831    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:51.406749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:51.406720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:51.506707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:51.506671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:51.610502 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:51.610469    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:55.011695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.011607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6" event={"ID":"2d01ea7b-c17c-4931-8f48-a21e74155ed1","Type":"ContainerStarted","Data":"68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f"}
Sep 11 18:17:55.012125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.011711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:17:55.049735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.049684    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6" podStartSLOduration=1.874720309 podStartE2EDuration="6.049667418s" totalImagesPullingTime="4.174947109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:17:55.04870832 +0000 UTC m=+443.641908876" watchObservedRunningTime="2026-09-11 18:17:55.049667418 +0000 UTC m=+443.642867974"
Sep 11 18:17:55.462451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.462417    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:55.570791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.570758    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:55.803864 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.803832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:55.918999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:55.918969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:57.490723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:57.490693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:57.985813 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:57.985776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:59.440250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:59.440216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:59.574102 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:17:59.574072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:04.260444 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:04.260409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:04.362997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:04.362968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:06.018846 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:06.018808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:08.183446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.183413    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"]
Sep 11 18:18:08.183902 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.183643    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" containerName="manager" containerID="cri-o://68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f" gracePeriod=2
Sep 11 18:18:08.196310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.196275    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"]
Sep 11 18:18:08.199600 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:08.199553    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2d01ea7b-c17c-4931-8f48-a21e74155ed1\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\": pods \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:08.219414 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.219379    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"]
Sep 11 18:18:08.219962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.219941    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" containerName="manager"
Sep 11 18:18:08.220003 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.219968    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" containerName="manager"
Sep 11 18:18:08.220101 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.220089    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" containerName="manager"
Sep 11 18:18:08.223583 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.223558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.225756 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:08.225719    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" is forbidden: User \"system:node:ip-10-0-140-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-142.ec2.internal' and this object" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:08.244882 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.244849    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"]
Sep 11 18:18:08.353058 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.353022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx6cr\" (UniqueName: \"kubernetes.io/projected/1d4ceceb-8df1-440f-bbb9-cec645b691e0-kube-api-access-hx6cr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sx4w\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.353242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.353076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4ceceb-8df1-440f-bbb9-cec645b691e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sx4w\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.434540 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.434467    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:08.436863 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:08.436831    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" is forbidden: User \"system:node:ip-10-0-140-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-142.ec2.internal' and this object" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:08.453845 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.453809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx6cr\" (UniqueName: \"kubernetes.io/projected/1d4ceceb-8df1-440f-bbb9-cec645b691e0-kube-api-access-hx6cr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sx4w\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.454014 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.453860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4ceceb-8df1-440f-bbb9-cec645b691e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sx4w\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.454255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.454234    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4ceceb-8df1-440f-bbb9-cec645b691e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sx4w\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.465364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.465335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx6cr\" (UniqueName: \"kubernetes.io/projected/1d4ceceb-8df1-440f-bbb9-cec645b691e0-kube-api-access-hx6cr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9sx4w\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.555091 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.555050    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hts6\" (UniqueName: \"kubernetes.io/projected/2d01ea7b-c17c-4931-8f48-a21e74155ed1-kube-api-access-7hts6\") pod \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") "
Sep 11 18:18:08.555230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.555143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d01ea7b-c17c-4931-8f48-a21e74155ed1-extensions-socket-volume\") pod \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\" (UID: \"2d01ea7b-c17c-4931-8f48-a21e74155ed1\") "
Sep 11 18:18:08.555647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.555622    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d01ea7b-c17c-4931-8f48-a21e74155ed1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2d01ea7b-c17c-4931-8f48-a21e74155ed1" (UID: "2d01ea7b-c17c-4931-8f48-a21e74155ed1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:08.557375 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.557343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d01ea7b-c17c-4931-8f48-a21e74155ed1-kube-api-access-7hts6" (OuterVolumeSpecName: "kube-api-access-7hts6") pod "2d01ea7b-c17c-4931-8f48-a21e74155ed1" (UID: "2d01ea7b-c17c-4931-8f48-a21e74155ed1"). InnerVolumeSpecName "kube-api-access-7hts6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:18:08.563444 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.563422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:08.655927 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.655883    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hts6\" (UniqueName: \"kubernetes.io/projected/2d01ea7b-c17c-4931-8f48-a21e74155ed1-kube-api-access-7hts6\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:18:08.655927 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.655922    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2d01ea7b-c17c-4931-8f48-a21e74155ed1-extensions-socket-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:18:08.848917 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:08.848884    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"]
Sep 11 18:18:08.851421 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:18:08.851386    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4ceceb_8df1_440f_bbb9_cec645b691e0.slice/crio-ee078da414cd45fe97338bc5e6fa5b198c2edb481bcd20e56aa495a98243b772 WatchSource:0}: Error finding container ee078da414cd45fe97338bc5e6fa5b198c2edb481bcd20e56aa495a98243b772: Status 404 returned error can't find the container with id ee078da414cd45fe97338bc5e6fa5b198c2edb481bcd20e56aa495a98243b772
Sep 11 18:18:09.077459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.077416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w" event={"ID":"1d4ceceb-8df1-440f-bbb9-cec645b691e0","Type":"ContainerStarted","Data":"e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7"}
Sep 11 18:18:09.077678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.077476    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w" event={"ID":"1d4ceceb-8df1-440f-bbb9-cec645b691e0","Type":"ContainerStarted","Data":"ee078da414cd45fe97338bc5e6fa5b198c2edb481bcd20e56aa495a98243b772"}
Sep 11 18:18:09.077678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.077503    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:09.078948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.078924    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" containerID="68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f" exitCode=0
Sep 11 18:18:09.079054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.078964    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:09.079054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.079017    2563 scope.go:117] "RemoveContainer" containerID="68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f"
Sep 11 18:18:09.081589 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:09.081553    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" is forbidden: User \"system:node:ip-10-0-140-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-142.ec2.internal' and this object" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:09.088241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.088221    2563 scope.go:117] "RemoveContainer" containerID="68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f"
Sep 11 18:18:09.088589 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:09.088568    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f\": container with ID starting with 68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f not found: ID does not exist" containerID="68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f"
Sep 11 18:18:09.088657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.088597    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f"} err="failed to get container status \"68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f\": rpc error: code = NotFound desc = could not find container \"68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f\": container with ID starting with 68f9709a61cedac13178e776dd0c28b38890c70f944526f55d0d4c227313230f not found: ID does not exist"
Sep 11 18:18:09.122253 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:09.122173    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-v6ff6\" is forbidden: User \"system:node:ip-10-0-140-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-142.ec2.internal' and this object" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-v6ff6"
Sep 11 18:18:09.917850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:09.917815    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d01ea7b-c17c-4931-8f48-a21e74155ed1" path="/var/lib/kubelet/pods/2d01ea7b-c17c-4931-8f48-a21e74155ed1/volumes"
Sep 11 18:18:11.051447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:11.051412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:11.161220 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:11.161181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:14.472707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:14.472673    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:14.551585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:14.551547    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:14.685905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:14.685863    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:14.778433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:14.778395    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:17.474975 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:17.474930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:17.580048 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:17.580006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:20.087248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:20.087214    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:20.111544 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:20.111492    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w" podStartSLOduration=12.111475122 podStartE2EDuration="12.111475122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:09.12023987 +0000 UTC m=+457.713440428" watchObservedRunningTime="2026-09-11 18:18:20.111475122 +0000 UTC m=+468.704675670"
Sep 11 18:18:23.425462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.420067    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"]
Sep 11 18:18:23.425462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.420402    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w" podUID="1d4ceceb-8df1-440f-bbb9-cec645b691e0" containerName="manager" containerID="cri-o://e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7" gracePeriod=10
Sep 11 18:18:23.665994 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.665971    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:23.689930 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.689873    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4ceceb-8df1-440f-bbb9-cec645b691e0-extensions-socket-volume\") pod \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") "
Sep 11 18:18:23.690054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.689914    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx6cr\" (UniqueName: \"kubernetes.io/projected/1d4ceceb-8df1-440f-bbb9-cec645b691e0-kube-api-access-hx6cr\") pod \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\" (UID: \"1d4ceceb-8df1-440f-bbb9-cec645b691e0\") "
Sep 11 18:18:23.690348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.690316    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4ceceb-8df1-440f-bbb9-cec645b691e0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1d4ceceb-8df1-440f-bbb9-cec645b691e0" (UID: "1d4ceceb-8df1-440f-bbb9-cec645b691e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:23.692685 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.692652    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4ceceb-8df1-440f-bbb9-cec645b691e0-kube-api-access-hx6cr" (OuterVolumeSpecName: "kube-api-access-hx6cr") pod "1d4ceceb-8df1-440f-bbb9-cec645b691e0" (UID: "1d4ceceb-8df1-440f-bbb9-cec645b691e0"). InnerVolumeSpecName "kube-api-access-hx6cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:18:23.791134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.791089    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4ceceb-8df1-440f-bbb9-cec645b691e0-extensions-socket-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:18:23.791134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:23.791126    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx6cr\" (UniqueName: \"kubernetes.io/projected/1d4ceceb-8df1-440f-bbb9-cec645b691e0-kube-api-access-hx6cr\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:18:24.059044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.058999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:24.152204 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.152170    2563 generic.go:356] "Generic (PLEG): container finished" podID="1d4ceceb-8df1-440f-bbb9-cec645b691e0" containerID="e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7" exitCode=0
Sep 11 18:18:24.152355 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.152234    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"
Sep 11 18:18:24.152355 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.152255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w" event={"ID":"1d4ceceb-8df1-440f-bbb9-cec645b691e0","Type":"ContainerDied","Data":"e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7"}
Sep 11 18:18:24.152355 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.152350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w" event={"ID":"1d4ceceb-8df1-440f-bbb9-cec645b691e0","Type":"ContainerDied","Data":"ee078da414cd45fe97338bc5e6fa5b198c2edb481bcd20e56aa495a98243b772"}
Sep 11 18:18:24.152489 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.152374    2563 scope.go:117] "RemoveContainer" containerID="e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7"
Sep 11 18:18:24.160666 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.160648    2563 scope.go:117] "RemoveContainer" containerID="e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7"
Sep 11 18:18:24.160962 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:18:24.160944    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7\": container with ID starting with e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7 not found: ID does not exist" containerID="e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7"
Sep 11 18:18:24.161007 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.160971    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7"} err="failed to get container status \"e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7\": rpc error: code = NotFound desc = could not find container \"e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7\": container with ID starting with e9dbbf0fb9835384b622617382e6ace1814b2fa3d6297d77208c0204a7b6f6f7 not found: ID does not exist"
Sep 11 18:18:24.177707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.177678    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"]
Sep 11 18:18:24.187741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.187718    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9sx4w"]
Sep 11 18:18:24.230915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:24.230882    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:25.912997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:25.912954    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4ceceb-8df1-440f-bbb9-cec645b691e0" path="/var/lib/kubelet/pods/1d4ceceb-8df1-440f-bbb9-cec645b691e0/volumes"
Sep 11 18:18:39.918413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.918382    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"]
Sep 11 18:18:39.918831 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.918688    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4ceceb-8df1-440f-bbb9-cec645b691e0" containerName="manager"
Sep 11 18:18:39.918831 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.918699    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4ceceb-8df1-440f-bbb9-cec645b691e0" containerName="manager"
Sep 11 18:18:39.918831 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.918774    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4ceceb-8df1-440f-bbb9-cec645b691e0" containerName="manager"
Sep 11 18:18:39.923480 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.923454    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:39.927137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.927115    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 18:18:39.927337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.927152    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hsrd5\""
Sep 11 18:18:39.947522 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.947499    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"]
Sep 11 18:18:39.987605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:39.987578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:40.032705 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed068df9-4d54-4e39-90a8-933b710e11c0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df2fg\" (UniqueName: \"kubernetes.io/projected/ed068df9-4d54-4e39-90a8-933b710e11c0-kube-api-access-df2fg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.032988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.032958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.033179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.033017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed068df9-4d54-4e39-90a8-933b710e11c0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df2fg\" (UniqueName: \"kubernetes.io/projected/ed068df9-4d54-4e39-90a8-933b710e11c0-kube-api-access-df2fg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.133942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133925    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.134074 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.133956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.134141 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.134092    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.134269 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.134225    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:40.134394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.134375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed068df9-4d54-4e39-90a8-933b710e11c0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.136141 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.136118    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.136248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.136232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.142228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.142206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed068df9-4d54-4e39-90a8-933b710e11c0-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.142957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.142933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df2fg\" (UniqueName: \"kubernetes.io/projected/ed068df9-4d54-4e39-90a8-933b710e11c0-kube-api-access-df2fg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-2gjdw\" (UID: \"ed068df9-4d54-4e39-90a8-933b710e11c0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.233838 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.233789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:40.511108 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:40.511081    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"]
Sep 11 18:18:40.513273 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:18:40.513224    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded068df9_4d54_4e39_90a8_933b710e11c0.slice/crio-fb99fc1f9c16dec5d10110c22333c7ed1cf1ada56cdade531e44871b9c0dfcc4 WatchSource:0}: Error finding container fb99fc1f9c16dec5d10110c22333c7ed1cf1ada56cdade531e44871b9c0dfcc4: Status 404 returned error can't find the container with id fb99fc1f9c16dec5d10110c22333c7ed1cf1ada56cdade531e44871b9c0dfcc4
Sep 11 18:18:41.231127 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:41.231090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" event={"ID":"ed068df9-4d54-4e39-90a8-933b710e11c0","Type":"ContainerStarted","Data":"fb99fc1f9c16dec5d10110c22333c7ed1cf1ada56cdade531e44871b9c0dfcc4"}
Sep 11 18:18:43.749679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:43.749635    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:18:43.749928 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:43.749717    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:18:43.749928 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:43.749752    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:18:44.246086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:44.246054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" event={"ID":"ed068df9-4d54-4e39-90a8-933b710e11c0","Type":"ContainerStarted","Data":"66dfe2f8585f11f5d700fcd5a5102c452473fd0bda817be0c14974406925f451"}
Sep 11 18:18:44.270293 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:44.270223    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" podStartSLOduration=2.036185886 podStartE2EDuration="5.270204376s" totalImagesPullingTime="3.23401849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:44.267186978 +0000 UTC m=+492.860387534" watchObservedRunningTime="2026-09-11 18:18:44.270204376 +0000 UTC m=+492.863404929"
Sep 11 18:18:45.234528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:45.234492    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:45.239253 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:45.239223    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:45.251820 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:45.251791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:45.252581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:45.252561    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:18:49.626491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:49.626459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:49.715219 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:49.715182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:55.606807 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.606773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:55.610661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.610632    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:18:55.614109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.614083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.616521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.616483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 18:18:55.616521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.616518    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 18:18:55.617203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.617179    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 18:18:55.617304 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.617181    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p2rph\""
Sep 11 18:18:55.624105 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.624074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:18:55.706661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.706626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:18:55.714666 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.714634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:55.778640 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.778586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:55.783873 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.783826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/921f0cb5-e8a2-479f-81a9-498e3126a546-config-file\") pod \"limitador-limitador-8588749977-bgqtt\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") " pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.783997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.783976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbhfx\" (UniqueName: \"kubernetes.io/projected/921f0cb5-e8a2-479f-81a9-498e3126a546-kube-api-access-vbhfx\") pod \"limitador-limitador-8588749977-bgqtt\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") " pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.885611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.885510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbhfx\" (UniqueName: \"kubernetes.io/projected/921f0cb5-e8a2-479f-81a9-498e3126a546-kube-api-access-vbhfx\") pod \"limitador-limitador-8588749977-bgqtt\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") " pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.885775 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.885625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/921f0cb5-e8a2-479f-81a9-498e3126a546-config-file\") pod \"limitador-limitador-8588749977-bgqtt\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") " pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.886388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.886363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/921f0cb5-e8a2-479f-81a9-498e3126a546-config-file\") pod \"limitador-limitador-8588749977-bgqtt\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") " pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.894895 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.894860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbhfx\" (UniqueName: \"kubernetes.io/projected/921f0cb5-e8a2-479f-81a9-498e3126a546-kube-api-access-vbhfx\") pod \"limitador-limitador-8588749977-bgqtt\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") " pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:55.925814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:55.925770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:56.202362 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.202337    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:18:56.204589 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:18:56.204562    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921f0cb5_e8a2_479f_81a9_498e3126a546.slice/crio-147e5c1e57a60ad616b4249240178ae4370805cdc91ba2c9bf0db4d5bb2c0039 WatchSource:0}: Error finding container 147e5c1e57a60ad616b4249240178ae4370805cdc91ba2c9bf0db4d5bb2c0039: Status 404 returned error can't find the container with id 147e5c1e57a60ad616b4249240178ae4370805cdc91ba2c9bf0db4d5bb2c0039
Sep 11 18:18:56.306271 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.306220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt" event={"ID":"921f0cb5-e8a2-479f-81a9-498e3126a546","Type":"ContainerStarted","Data":"147e5c1e57a60ad616b4249240178ae4370805cdc91ba2c9bf0db4d5bb2c0039"}
Sep 11 18:18:56.415057 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.415014    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hsfrk"]
Sep 11 18:18:56.420106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.420074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:18:56.422721 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.422690    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8mvdb\""
Sep 11 18:18:56.427481 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.427449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hsfrk"]
Sep 11 18:18:56.451880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.451802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:56.528245 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.528209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:56.531715 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.531680    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-xf67q"]
Sep 11 18:18:56.535124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.535102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:18:56.543898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.543858    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xf67q"]
Sep 11 18:18:56.593290 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.593220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xb72\" (UniqueName: \"kubernetes.io/projected/d712b215-7000-4f9b-a054-29c32769f53f-kube-api-access-6xb72\") pod \"authorino-7bcfd8f9cd-hsfrk\" (UID: \"d712b215-7000-4f9b-a054-29c32769f53f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:18:56.694382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.694339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2kqm\" (UniqueName: \"kubernetes.io/projected/cee8afdc-7167-4bfa-9be6-149119a3a1f1-kube-api-access-r2kqm\") pod \"authorino-59c974bd48-xf67q\" (UID: \"cee8afdc-7167-4bfa-9be6-149119a3a1f1\") " pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:18:56.694855 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.694464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xb72\" (UniqueName: \"kubernetes.io/projected/d712b215-7000-4f9b-a054-29c32769f53f-kube-api-access-6xb72\") pod \"authorino-7bcfd8f9cd-hsfrk\" (UID: \"d712b215-7000-4f9b-a054-29c32769f53f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:18:56.703415 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.703337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xb72\" (UniqueName: \"kubernetes.io/projected/d712b215-7000-4f9b-a054-29c32769f53f-kube-api-access-6xb72\") pod \"authorino-7bcfd8f9cd-hsfrk\" (UID: \"d712b215-7000-4f9b-a054-29c32769f53f\") " pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:18:56.739084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.739034    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:18:56.796179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.795892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2kqm\" (UniqueName: \"kubernetes.io/projected/cee8afdc-7167-4bfa-9be6-149119a3a1f1-kube-api-access-r2kqm\") pod \"authorino-59c974bd48-xf67q\" (UID: \"cee8afdc-7167-4bfa-9be6-149119a3a1f1\") " pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:18:56.805028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.804985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2kqm\" (UniqueName: \"kubernetes.io/projected/cee8afdc-7167-4bfa-9be6-149119a3a1f1-kube-api-access-r2kqm\") pod \"authorino-59c974bd48-xf67q\" (UID: \"cee8afdc-7167-4bfa-9be6-149119a3a1f1\") " pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:18:56.844800 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:56.844756    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:18:57.012066 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:57.011895    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hsfrk"]
Sep 11 18:18:57.014992 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:18:57.014952    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd712b215_7000_4f9b_a054_29c32769f53f.slice/crio-ac4e9cd5d5045a2107339f6084a80edf2759742fa28a0d15f643486ccf06506b WatchSource:0}: Error finding container ac4e9cd5d5045a2107339f6084a80edf2759742fa28a0d15f643486ccf06506b: Status 404 returned error can't find the container with id ac4e9cd5d5045a2107339f6084a80edf2759742fa28a0d15f643486ccf06506b
Sep 11 18:18:57.116884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:57.116855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xf67q"]
Sep 11 18:18:57.260169 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:18:57.260129    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcee8afdc_7167_4bfa_9be6_149119a3a1f1.slice/crio-f00c1d1637df0e563cd8fa753893b365908823bce118d65e385da69368c7cafb WatchSource:0}: Error finding container f00c1d1637df0e563cd8fa753893b365908823bce118d65e385da69368c7cafb: Status 404 returned error can't find the container with id f00c1d1637df0e563cd8fa753893b365908823bce118d65e385da69368c7cafb
Sep 11 18:18:57.312638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:57.312603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk" event={"ID":"d712b215-7000-4f9b-a054-29c32769f53f","Type":"ContainerStarted","Data":"ac4e9cd5d5045a2107339f6084a80edf2759742fa28a0d15f643486ccf06506b"}
Sep 11 18:18:57.313720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:57.313694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xf67q" event={"ID":"cee8afdc-7167-4bfa-9be6-149119a3a1f1","Type":"ContainerStarted","Data":"f00c1d1637df0e563cd8fa753893b365908823bce118d65e385da69368c7cafb"}
Sep 11 18:18:58.321636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:58.321600    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt" event={"ID":"921f0cb5-e8a2-479f-81a9-498e3126a546","Type":"ContainerStarted","Data":"e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809"}
Sep 11 18:18:58.322241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:58.321696    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:18:58.345038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:58.344987    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt" podStartSLOduration=2.226128691 podStartE2EDuration="3.34497088s" totalImagesPullingTime="1.118842189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:58.343689775 +0000 UTC m=+506.936890333" watchObservedRunningTime="2026-09-11 18:18:58.34497088 +0000 UTC m=+506.938171437"
Sep 11 18:18:59.327364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:59.327326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk" event={"ID":"d712b215-7000-4f9b-a054-29c32769f53f","Type":"ContainerStarted","Data":"965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582"}
Sep 11 18:18:59.344197 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:18:59.344142    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk" podStartSLOduration=1.368311256 podStartE2EDuration="3.344124381s" totalImagesPullingTime="1.975813125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:59.342685819 +0000 UTC m=+507.935886376" watchObservedRunningTime="2026-09-11 18:18:59.344124381 +0000 UTC m=+507.937324937"
Sep 11 18:19:00.333074 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:00.333039    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xf67q" event={"ID":"cee8afdc-7167-4bfa-9be6-149119a3a1f1","Type":"ContainerStarted","Data":"043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03"}
Sep 11 18:19:00.349365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:00.349320    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-xf67q" podStartSLOduration=2.125408942 podStartE2EDuration="4.349305186s" totalImagesPullingTime="2.223896244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:00.348369955 +0000 UTC m=+508.941570514" watchObservedRunningTime="2026-09-11 18:19:00.349305186 +0000 UTC m=+508.942505743"
Sep 11 18:19:00.375722 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:00.375695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hsfrk"]
Sep 11 18:19:01.337667 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:01.337631    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk" podUID="d712b215-7000-4f9b-a054-29c32769f53f" containerName="authorino" containerID="cri-o://965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582" gracePeriod=30
Sep 11 18:19:01.582612 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:01.582590    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:19:01.639382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:01.639316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xb72\" (UniqueName: \"kubernetes.io/projected/d712b215-7000-4f9b-a054-29c32769f53f-kube-api-access-6xb72\") pod \"d712b215-7000-4f9b-a054-29c32769f53f\" (UID: \"d712b215-7000-4f9b-a054-29c32769f53f\") "
Sep 11 18:19:01.641172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:01.641152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d712b215-7000-4f9b-a054-29c32769f53f-kube-api-access-6xb72" (OuterVolumeSpecName: "kube-api-access-6xb72") pod "d712b215-7000-4f9b-a054-29c32769f53f" (UID: "d712b215-7000-4f9b-a054-29c32769f53f"). InnerVolumeSpecName "kube-api-access-6xb72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:01.740305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:01.740280    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xb72\" (UniqueName: \"kubernetes.io/projected/d712b215-7000-4f9b-a054-29c32769f53f-kube-api-access-6xb72\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:02.343514 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.343479    2563 generic.go:356] "Generic (PLEG): container finished" podID="d712b215-7000-4f9b-a054-29c32769f53f" containerID="965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582" exitCode=0
Sep 11 18:19:02.343915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.343519    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk"
Sep 11 18:19:02.343915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.343540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk" event={"ID":"d712b215-7000-4f9b-a054-29c32769f53f","Type":"ContainerDied","Data":"965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582"}
Sep 11 18:19:02.343915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.343586    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-hsfrk" event={"ID":"d712b215-7000-4f9b-a054-29c32769f53f","Type":"ContainerDied","Data":"ac4e9cd5d5045a2107339f6084a80edf2759742fa28a0d15f643486ccf06506b"}
Sep 11 18:19:02.343915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.343613    2563 scope.go:117] "RemoveContainer" containerID="965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582"
Sep 11 18:19:02.352427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.352411    2563 scope.go:117] "RemoveContainer" containerID="965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582"
Sep 11 18:19:02.352709 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:02.352690    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582\": container with ID starting with 965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582 not found: ID does not exist" containerID="965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582"
Sep 11 18:19:02.352766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.352717    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582"} err="failed to get container status \"965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582\": rpc error: code = NotFound desc = could not find container \"965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582\": container with ID starting with 965360ed066dd5d7b2abc7162b80f2cf263f0a7ad8388ecfe6f5a4e126bfd582 not found: ID does not exist"
Sep 11 18:19:02.360449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.360429    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hsfrk"]
Sep 11 18:19:02.365187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:02.365168    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-hsfrk"]
Sep 11 18:19:03.913387 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:03.913351    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d712b215-7000-4f9b-a054-29c32769f53f" path="/var/lib/kubelet/pods/d712b215-7000-4f9b-a054-29c32769f53f/volumes"
Sep 11 18:19:09.017823 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.017787    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:19:09.018229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.017971    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt" podUID="921f0cb5-e8a2-479f-81a9-498e3126a546" containerName="limitador" containerID="cri-o://e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809" gracePeriod=30
Sep 11 18:19:09.018814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.018757    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:19:09.564830 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.564803    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:19:09.593731 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.593658    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/921f0cb5-e8a2-479f-81a9-498e3126a546-config-file\") pod \"921f0cb5-e8a2-479f-81a9-498e3126a546\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") "
Sep 11 18:19:09.593731 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.593713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbhfx\" (UniqueName: \"kubernetes.io/projected/921f0cb5-e8a2-479f-81a9-498e3126a546-kube-api-access-vbhfx\") pod \"921f0cb5-e8a2-479f-81a9-498e3126a546\" (UID: \"921f0cb5-e8a2-479f-81a9-498e3126a546\") "
Sep 11 18:19:09.594046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.594023    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/921f0cb5-e8a2-479f-81a9-498e3126a546-config-file" (OuterVolumeSpecName: "config-file") pod "921f0cb5-e8a2-479f-81a9-498e3126a546" (UID: "921f0cb5-e8a2-479f-81a9-498e3126a546"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:19:09.595863 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.595830    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/921f0cb5-e8a2-479f-81a9-498e3126a546-kube-api-access-vbhfx" (OuterVolumeSpecName: "kube-api-access-vbhfx") pod "921f0cb5-e8a2-479f-81a9-498e3126a546" (UID: "921f0cb5-e8a2-479f-81a9-498e3126a546"). InnerVolumeSpecName "kube-api-access-vbhfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:09.694633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.694595    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/921f0cb5-e8a2-479f-81a9-498e3126a546-config-file\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:09.694633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:09.694629    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbhfx\" (UniqueName: \"kubernetes.io/projected/921f0cb5-e8a2-479f-81a9-498e3126a546-kube-api-access-vbhfx\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:10.384896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.384862    2563 generic.go:356] "Generic (PLEG): container finished" podID="921f0cb5-e8a2-479f-81a9-498e3126a546" containerID="e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809" exitCode=0
Sep 11 18:19:10.385288 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.384926    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt"
Sep 11 18:19:10.385288 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.384945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt" event={"ID":"921f0cb5-e8a2-479f-81a9-498e3126a546","Type":"ContainerDied","Data":"e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809"}
Sep 11 18:19:10.385288 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.385007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-bgqtt" event={"ID":"921f0cb5-e8a2-479f-81a9-498e3126a546","Type":"ContainerDied","Data":"147e5c1e57a60ad616b4249240178ae4370805cdc91ba2c9bf0db4d5bb2c0039"}
Sep 11 18:19:10.385288 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.385036    2563 scope.go:117] "RemoveContainer" containerID="e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809"
Sep 11 18:19:10.393509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.393489    2563 scope.go:117] "RemoveContainer" containerID="e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809"
Sep 11 18:19:10.393809 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:10.393787    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809\": container with ID starting with e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809 not found: ID does not exist" containerID="e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809"
Sep 11 18:19:10.393856 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.393818    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809"} err="failed to get container status \"e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809\": rpc error: code = NotFound desc = could not find container \"e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809\": container with ID starting with e7a8340e1c409575d88f8a78b1abc2dd07d8971b5961d19655e91ad109b48809 not found: ID does not exist"
Sep 11 18:19:10.403409 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.403376    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:19:10.407540 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:10.407516    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-bgqtt"]
Sep 11 18:19:11.758174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:11.758140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:11.908328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:11.908272    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:11.913082 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:11.913054    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="921f0cb5-e8a2-479f-81a9-498e3126a546" path="/var/lib/kubelet/pods/921f0cb5-e8a2-479f-81a9-498e3126a546/volumes"
Sep 11 18:19:12.200304 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:12.200248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:24.042312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.042198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:24.266283 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266214    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-k662j"]
Sep 11 18:19:24.266636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266622    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d712b215-7000-4f9b-a054-29c32769f53f" containerName="authorino"
Sep 11 18:19:24.266679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266638    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d712b215-7000-4f9b-a054-29c32769f53f" containerName="authorino"
Sep 11 18:19:24.266679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266649    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="921f0cb5-e8a2-479f-81a9-498e3126a546" containerName="limitador"
Sep 11 18:19:24.266679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266655    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="921f0cb5-e8a2-479f-81a9-498e3126a546" containerName="limitador"
Sep 11 18:19:24.266778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266720    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="921f0cb5-e8a2-479f-81a9-498e3126a546" containerName="limitador"
Sep 11 18:19:24.266778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.266727    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d712b215-7000-4f9b-a054-29c32769f53f" containerName="authorino"
Sep 11 18:19:24.275576 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.275550    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:24.282236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.282199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-k662j"]
Sep 11 18:19:24.286645 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.286608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:24.332799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.332702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hs8k\" (UniqueName: \"kubernetes.io/projected/438655ae-6d2c-441f-852c-f545c4724e79-kube-api-access-4hs8k\") pod \"authorino-557f876f69-k662j\" (UID: \"438655ae-6d2c-441f-852c-f545c4724e79\") " pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:24.402648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.402608    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-k662j"]
Sep 11 18:19:24.402921 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:24.402898    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4hs8k], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-k662j" podUID="438655ae-6d2c-441f-852c-f545c4724e79"
Sep 11 18:19:24.433567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.433524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hs8k\" (UniqueName: \"kubernetes.io/projected/438655ae-6d2c-441f-852c-f545c4724e79-kube-api-access-4hs8k\") pod \"authorino-557f876f69-k662j\" (UID: \"438655ae-6d2c-441f-852c-f545c4724e79\") " pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:24.434100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.434080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-gsj4d"]
Sep 11 18:19:24.437567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.437549    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.439637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.439614    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 18:19:24.445488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.445463    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-gsj4d"]
Sep 11 18:19:24.446779 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.446757    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hs8k\" (UniqueName: \"kubernetes.io/projected/438655ae-6d2c-441f-852c-f545c4724e79-kube-api-access-4hs8k\") pod \"authorino-557f876f69-k662j\" (UID: \"438655ae-6d2c-441f-852c-f545c4724e79\") " pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:24.456442 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.456413    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:24.465137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.465107    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:24.519931 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.519897    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-gsj4d"]
Sep 11 18:19:24.520209 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:24.520187    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-v8grl tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-gsj4d" podUID="0f64ce99-0c52-4079-9046-d6f68bd8de51"
Sep 11 18:19:24.534591 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.534559    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hs8k\" (UniqueName: \"kubernetes.io/projected/438655ae-6d2c-441f-852c-f545c4724e79-kube-api-access-4hs8k\") pod \"438655ae-6d2c-441f-852c-f545c4724e79\" (UID: \"438655ae-6d2c-441f-852c-f545c4724e79\") "
Sep 11 18:19:24.534734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.534720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f64ce99-0c52-4079-9046-d6f68bd8de51-tls-cert\") pod \"authorino-6d4dfc649f-gsj4d\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") " pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.534778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.534762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8grl\" (UniqueName: \"kubernetes.io/projected/0f64ce99-0c52-4079-9046-d6f68bd8de51-kube-api-access-v8grl\") pod \"authorino-6d4dfc649f-gsj4d\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") " pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.536720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.536693    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/438655ae-6d2c-441f-852c-f545c4724e79-kube-api-access-4hs8k" (OuterVolumeSpecName: "kube-api-access-4hs8k") pod "438655ae-6d2c-441f-852c-f545c4724e79" (UID: "438655ae-6d2c-441f-852c-f545c4724e79"). InnerVolumeSpecName "kube-api-access-4hs8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:24.548864 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.548838    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76b96f6658-7gsx6"]
Sep 11 18:19:24.552143 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.552124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.560347 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.560324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b96f6658-7gsx6"]
Sep 11 18:19:24.635903 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.635806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86fhl\" (UniqueName: \"kubernetes.io/projected/d3187c01-70bb-4717-a092-62a0873474ff-kube-api-access-86fhl\") pod \"authorino-76b96f6658-7gsx6\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") " pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.635903 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.635856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f64ce99-0c52-4079-9046-d6f68bd8de51-tls-cert\") pod \"authorino-6d4dfc649f-gsj4d\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") " pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.636113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.635939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8grl\" (UniqueName: \"kubernetes.io/projected/0f64ce99-0c52-4079-9046-d6f68bd8de51-kube-api-access-v8grl\") pod \"authorino-6d4dfc649f-gsj4d\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") " pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.636113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.635973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3187c01-70bb-4717-a092-62a0873474ff-tls-cert\") pod \"authorino-76b96f6658-7gsx6\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") " pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.636113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.636043    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hs8k\" (UniqueName: \"kubernetes.io/projected/438655ae-6d2c-441f-852c-f545c4724e79-kube-api-access-4hs8k\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:24.638298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.638251    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f64ce99-0c52-4079-9046-d6f68bd8de51-tls-cert\") pod \"authorino-6d4dfc649f-gsj4d\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") " pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.644516 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.644488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8grl\" (UniqueName: \"kubernetes.io/projected/0f64ce99-0c52-4079-9046-d6f68bd8de51-kube-api-access-v8grl\") pod \"authorino-6d4dfc649f-gsj4d\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") " pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:24.737085 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.737038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86fhl\" (UniqueName: \"kubernetes.io/projected/d3187c01-70bb-4717-a092-62a0873474ff-kube-api-access-86fhl\") pod \"authorino-76b96f6658-7gsx6\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") " pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.737299 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.737122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3187c01-70bb-4717-a092-62a0873474ff-tls-cert\") pod \"authorino-76b96f6658-7gsx6\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") " pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.739558 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.739536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3187c01-70bb-4717-a092-62a0873474ff-tls-cert\") pod \"authorino-76b96f6658-7gsx6\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") " pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.746040 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.746010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86fhl\" (UniqueName: \"kubernetes.io/projected/d3187c01-70bb-4717-a092-62a0873474ff-kube-api-access-86fhl\") pod \"authorino-76b96f6658-7gsx6\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") " pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:24.862415 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:24.862371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:19:25.132795 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.132767    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b96f6658-7gsx6"]
Sep 11 18:19:25.135336 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:19:25.135308    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3187c01_70bb_4717_a092_62a0873474ff.slice/crio-eddebd3b6396fea91141b984530e57ea7ce778e76701c01cdfa11bdc0ec41970 WatchSource:0}: Error finding container eddebd3b6396fea91141b984530e57ea7ce778e76701c01cdfa11bdc0ec41970: Status 404 returned error can't find the container with id eddebd3b6396fea91141b984530e57ea7ce778e76701c01cdfa11bdc0ec41970
Sep 11 18:19:25.462332 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.462293    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b96f6658-7gsx6" event={"ID":"d3187c01-70bb-4717-a092-62a0873474ff","Type":"ContainerStarted","Data":"eddebd3b6396fea91141b984530e57ea7ce778e76701c01cdfa11bdc0ec41970"}
Sep 11 18:19:25.462493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.462372    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-k662j"
Sep 11 18:19:25.462493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.462372    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:25.470802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.470781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:25.504632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.504585    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-k662j"]
Sep 11 18:19:25.508703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.508673    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-k662j"]
Sep 11 18:19:25.546100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.546069    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f64ce99-0c52-4079-9046-d6f68bd8de51-tls-cert\") pod \"0f64ce99-0c52-4079-9046-d6f68bd8de51\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") "
Sep 11 18:19:25.546331 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.546166    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8grl\" (UniqueName: \"kubernetes.io/projected/0f64ce99-0c52-4079-9046-d6f68bd8de51-kube-api-access-v8grl\") pod \"0f64ce99-0c52-4079-9046-d6f68bd8de51\" (UID: \"0f64ce99-0c52-4079-9046-d6f68bd8de51\") "
Sep 11 18:19:25.548249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.548213    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f64ce99-0c52-4079-9046-d6f68bd8de51-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0f64ce99-0c52-4079-9046-d6f68bd8de51" (UID: "0f64ce99-0c52-4079-9046-d6f68bd8de51"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:19:25.548249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.548213    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f64ce99-0c52-4079-9046-d6f68bd8de51-kube-api-access-v8grl" (OuterVolumeSpecName: "kube-api-access-v8grl") pod "0f64ce99-0c52-4079-9046-d6f68bd8de51" (UID: "0f64ce99-0c52-4079-9046-d6f68bd8de51"). InnerVolumeSpecName "kube-api-access-v8grl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:25.647230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.647193    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8grl\" (UniqueName: \"kubernetes.io/projected/0f64ce99-0c52-4079-9046-d6f68bd8de51-kube-api-access-v8grl\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:25.647230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.647224    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f64ce99-0c52-4079-9046-d6f68bd8de51-tls-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:25.913870 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:25.913736    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="438655ae-6d2c-441f-852c-f545c4724e79" path="/var/lib/kubelet/pods/438655ae-6d2c-441f-852c-f545c4724e79/volumes"
Sep 11 18:19:26.469216 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.469187    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-gsj4d"
Sep 11 18:19:26.469643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.469178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b96f6658-7gsx6" event={"ID":"d3187c01-70bb-4717-a092-62a0873474ff","Type":"ContainerStarted","Data":"439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79"}
Sep 11 18:19:26.489212 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.489155    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76b96f6658-7gsx6" podStartSLOduration=1.818359616 podStartE2EDuration="2.489137975s" totalImagesPullingTime="670.778359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:26.487377205 +0000 UTC m=+535.080577774" watchObservedRunningTime="2026-09-11 18:19:26.489137975 +0000 UTC m=+535.082338532"
Sep 11 18:19:26.513028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.512992    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-gsj4d"]
Sep 11 18:19:26.520039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.519999    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xf67q"]
Sep 11 18:19:26.520300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.520236    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-xf67q" podUID="cee8afdc-7167-4bfa-9be6-149119a3a1f1" containerName="authorino" containerID="cri-o://043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03" gracePeriod=30
Sep 11 18:19:26.521745 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.521720    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-gsj4d"]
Sep 11 18:19:26.763793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.763763    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:19:26.858000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.857957    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2kqm\" (UniqueName: \"kubernetes.io/projected/cee8afdc-7167-4bfa-9be6-149119a3a1f1-kube-api-access-r2kqm\") pod \"cee8afdc-7167-4bfa-9be6-149119a3a1f1\" (UID: \"cee8afdc-7167-4bfa-9be6-149119a3a1f1\") "
Sep 11 18:19:26.860103 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.860074    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cee8afdc-7167-4bfa-9be6-149119a3a1f1-kube-api-access-r2kqm" (OuterVolumeSpecName: "kube-api-access-r2kqm") pod "cee8afdc-7167-4bfa-9be6-149119a3a1f1" (UID: "cee8afdc-7167-4bfa-9be6-149119a3a1f1"). InnerVolumeSpecName "kube-api-access-r2kqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:26.959179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:26.959140    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2kqm\" (UniqueName: \"kubernetes.io/projected/cee8afdc-7167-4bfa-9be6-149119a3a1f1-kube-api-access-r2kqm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:27.475733 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.475694    2563 generic.go:356] "Generic (PLEG): container finished" podID="cee8afdc-7167-4bfa-9be6-149119a3a1f1" containerID="043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03" exitCode=0
Sep 11 18:19:27.476209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.475748    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-xf67q"
Sep 11 18:19:27.476209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.475785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xf67q" event={"ID":"cee8afdc-7167-4bfa-9be6-149119a3a1f1","Type":"ContainerDied","Data":"043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03"}
Sep 11 18:19:27.476209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.475830    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-xf67q" event={"ID":"cee8afdc-7167-4bfa-9be6-149119a3a1f1","Type":"ContainerDied","Data":"f00c1d1637df0e563cd8fa753893b365908823bce118d65e385da69368c7cafb"}
Sep 11 18:19:27.476209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.475848    2563 scope.go:117] "RemoveContainer" containerID="043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03"
Sep 11 18:19:27.485481 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.485457    2563 scope.go:117] "RemoveContainer" containerID="043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03"
Sep 11 18:19:27.485829 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:27.485808    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03\": container with ID starting with 043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03 not found: ID does not exist" containerID="043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03"
Sep 11 18:19:27.485880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.485839    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03"} err="failed to get container status \"043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03\": rpc error: code = NotFound desc = could not find container \"043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03\": container with ID starting with 043a77b9c2399a906f035cef22209252b759895f0f5368224ae37d10efc97d03 not found: ID does not exist"
Sep 11 18:19:27.496852 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.496818    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xf67q"]
Sep 11 18:19:27.499107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.499078    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-xf67q"]
Sep 11 18:19:27.912888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.912856    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f64ce99-0c52-4079-9046-d6f68bd8de51" path="/var/lib/kubelet/pods/0f64ce99-0c52-4079-9046-d6f68bd8de51/volumes"
Sep 11 18:19:27.913105 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:27.913093    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cee8afdc-7167-4bfa-9be6-149119a3a1f1" path="/var/lib/kubelet/pods/cee8afdc-7167-4bfa-9be6-149119a3a1f1/volumes"
Sep 11 18:19:31.207460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.207428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:31.248020 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.247981    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57bcb96654-bqf94"]
Sep 11 18:19:31.248538 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.248519    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee8afdc-7167-4bfa-9be6-149119a3a1f1" containerName="authorino"
Sep 11 18:19:31.248538 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.248540    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee8afdc-7167-4bfa-9be6-149119a3a1f1" containerName="authorino"
Sep 11 18:19:31.248712 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.248676    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cee8afdc-7167-4bfa-9be6-149119a3a1f1" containerName="authorino"
Sep 11 18:19:31.252084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.252056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.254367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.254339    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 18:19:31.254537 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.254512    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 18:19:31.254588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.254521    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mh8vn\""
Sep 11 18:19:31.255156 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.255138    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 18:19:31.261369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.261337    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57bcb96654-bqf94"]
Sep 11 18:19:31.279736 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.279653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:31.339934 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.339903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:31.399388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.399350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.399618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.399501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.399618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.399542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrbqj\" (UniqueName: \"kubernetes.io/projected/46610332-c630-4581-aebb-4e88e3c9197d-kube-api-access-mrbqj\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.420147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.420110    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-556697db7-qfjsc"]
Sep 11 18:19:31.422743 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.422717    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.433498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.433452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-556697db7-qfjsc"]
Sep 11 18:19:31.500737 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.500639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7zbm\" (UniqueName: \"kubernetes.io/projected/71633f76-16bd-4065-8521-af0bdb28a5d1-kube-api-access-s7zbm\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.500737 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.500698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.500971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.500778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.500971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.500817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrbqj\" (UniqueName: \"kubernetes.io/projected/46610332-c630-4581-aebb-4e88e3c9197d-kube-api-access-mrbqj\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.500971 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.500884    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 11 18:19:31.500971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.500894    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.500971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.500921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.500971 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.500953    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs podName:46610332-c630-4581-aebb-4e88e3c9197d nodeName:}" failed. No retries permitted until 2026-09-11 18:19:32.000931612 +0000 UTC m=+540.594132171 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs") pod "maas-controller-57bcb96654-bqf94" (UID: "46610332-c630-4581-aebb-4e88e3c9197d") : secret "maas-controller-metrics-tls" not found
Sep 11 18:19:31.501275 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.501019    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 18:19:31.501275 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.501071    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert podName:46610332-c630-4581-aebb-4e88e3c9197d nodeName:}" failed. No retries permitted until 2026-09-11 18:19:32.00105522 +0000 UTC m=+540.594255765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert") pod "maas-controller-57bcb96654-bqf94" (UID: "46610332-c630-4581-aebb-4e88e3c9197d") : secret "maas-controller-webhook-cert" not found
Sep 11 18:19:31.511791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.511764    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrbqj\" (UniqueName: \"kubernetes.io/projected/46610332-c630-4581-aebb-4e88e3c9197d-kube-api-access-mrbqj\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:31.601810 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.601764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.602022 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.601831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zbm\" (UniqueName: \"kubernetes.io/projected/71633f76-16bd-4065-8521-af0bdb28a5d1-kube-api-access-s7zbm\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.602022 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.601865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.602022 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.601959    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 11 18:19:31.602022 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.601994    2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 11 18:19:31.602197 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.602046    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs podName:71633f76-16bd-4065-8521-af0bdb28a5d1 nodeName:}" failed. No retries permitted until 2026-09-11 18:19:32.102024429 +0000 UTC m=+540.695224967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs") pod "maas-controller-556697db7-qfjsc" (UID: "71633f76-16bd-4065-8521-af0bdb28a5d1") : secret "maas-controller-metrics-tls" not found
Sep 11 18:19:31.602197 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:31.602065    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert podName:71633f76-16bd-4065-8521-af0bdb28a5d1 nodeName:}" failed. No retries permitted until 2026-09-11 18:19:32.102056083 +0000 UTC m=+540.695256627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert") pod "maas-controller-556697db7-qfjsc" (UID: "71633f76-16bd-4065-8521-af0bdb28a5d1") : secret "maas-controller-webhook-cert" not found
Sep 11 18:19:31.610339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.610304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7zbm\" (UniqueName: \"kubernetes.io/projected/71633f76-16bd-4065-8521-af0bdb28a5d1-kube-api-access-s7zbm\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:31.727801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.727765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:31.953199 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:31.953166    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:32.005521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.005488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:32.005720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.005594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:32.008069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.008042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:32.008069 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.008062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert\") pod \"maas-controller-57bcb96654-bqf94\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") " pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:32.106694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.106595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:32.106694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.106659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:32.109143 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.109120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:32.109239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.109162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs\") pod \"maas-controller-556697db7-qfjsc\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") " pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:32.167080 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.167045    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mh8vn\""
Sep 11 18:19:32.175831 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.175801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:32.335173 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.335127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:32.449969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.449945    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57bcb96654-bqf94"]
Sep 11 18:19:32.451820 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:19:32.451787    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46610332_c630_4581_aebb_4e88e3c9197d.slice/crio-5f66631a128b70faf2a4abc4ebd4ae0493c7aa9039f04732d8a7938ac3f177d8 WatchSource:0}: Error finding container 5f66631a128b70faf2a4abc4ebd4ae0493c7aa9039f04732d8a7938ac3f177d8: Status 404 returned error can't find the container with id 5f66631a128b70faf2a4abc4ebd4ae0493c7aa9039f04732d8a7938ac3f177d8
Sep 11 18:19:32.501583 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.501547    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57bcb96654-bqf94" event={"ID":"46610332-c630-4581-aebb-4e88e3c9197d","Type":"ContainerStarted","Data":"5f66631a128b70faf2a4abc4ebd4ae0493c7aa9039f04732d8a7938ac3f177d8"}
Sep 11 18:19:32.606935 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:32.606905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-556697db7-qfjsc"]
Sep 11 18:19:32.608938 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:19:32.608910    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71633f76_16bd_4065_8521_af0bdb28a5d1.slice/crio-631edab59de8fa9392b8d4192cb8c1f1f80c3d774f8665020ec6b60fbc10f787 WatchSource:0}: Error finding container 631edab59de8fa9392b8d4192cb8c1f1f80c3d774f8665020ec6b60fbc10f787: Status 404 returned error can't find the container with id 631edab59de8fa9392b8d4192cb8c1f1f80c3d774f8665020ec6b60fbc10f787
Sep 11 18:19:33.512049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:33.512011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-qfjsc" event={"ID":"71633f76-16bd-4065-8521-af0bdb28a5d1","Type":"ContainerStarted","Data":"631edab59de8fa9392b8d4192cb8c1f1f80c3d774f8665020ec6b60fbc10f787"}
Sep 11 18:19:35.057435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.057404    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 18:19:35.263629 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.263595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:35.369667 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.369590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:35.523898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.523863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57bcb96654-bqf94" event={"ID":"46610332-c630-4581-aebb-4e88e3c9197d","Type":"ContainerStarted","Data":"6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de"}
Sep 11 18:19:35.524095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.524002    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:35.525713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.525687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-qfjsc" event={"ID":"71633f76-16bd-4065-8521-af0bdb28a5d1","Type":"ContainerStarted","Data":"6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44"}
Sep 11 18:19:35.525839 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.525822    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:35.543794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.543753    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57bcb96654-bqf94" podStartSLOduration=1.941989467 podStartE2EDuration="4.543741212s" totalImagesPullingTime="2.601751745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:35.542663701 +0000 UTC m=+544.135864260" watchObservedRunningTime="2026-09-11 18:19:35.543741212 +0000 UTC m=+544.136941769"
Sep 11 18:19:35.563546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:35.563506    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-556697db7-qfjsc" podStartSLOduration=2.112168449 podStartE2EDuration="4.563493038s" totalImagesPullingTime="2.451324589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:35.562012501 +0000 UTC m=+544.155213058" watchObservedRunningTime="2026-09-11 18:19:35.563493038 +0000 UTC m=+544.156693641"
Sep 11 18:19:46.533701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.533669    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:46.534126 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.533911    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:19:46.611057 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.611022    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57bcb96654-bqf94"]
Sep 11 18:19:46.611279 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.611231    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57bcb96654-bqf94" podUID="46610332-c630-4581-aebb-4e88e3c9197d" containerName="manager" containerID="cri-o://6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de" gracePeriod=30
Sep 11 18:19:46.855136 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.855113    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:46.941154 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.941125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs\") pod \"46610332-c630-4581-aebb-4e88e3c9197d\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") "
Sep 11 18:19:46.941318 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.941201    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrbqj\" (UniqueName: \"kubernetes.io/projected/46610332-c630-4581-aebb-4e88e3c9197d-kube-api-access-mrbqj\") pod \"46610332-c630-4581-aebb-4e88e3c9197d\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") "
Sep 11 18:19:46.941318 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.941247    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert\") pod \"46610332-c630-4581-aebb-4e88e3c9197d\" (UID: \"46610332-c630-4581-aebb-4e88e3c9197d\") "
Sep 11 18:19:46.943453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.943419    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46610332-c630-4581-aebb-4e88e3c9197d-kube-api-access-mrbqj" (OuterVolumeSpecName: "kube-api-access-mrbqj") pod "46610332-c630-4581-aebb-4e88e3c9197d" (UID: "46610332-c630-4581-aebb-4e88e3c9197d"). InnerVolumeSpecName "kube-api-access-mrbqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:19:46.943453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.943443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46610332-c630-4581-aebb-4e88e3c9197d" (UID: "46610332-c630-4581-aebb-4e88e3c9197d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:19:46.943585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:46.943501    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "46610332-c630-4581-aebb-4e88e3c9197d" (UID: "46610332-c630-4581-aebb-4e88e3c9197d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:19:47.042049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.042025    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrbqj\" (UniqueName: \"kubernetes.io/projected/46610332-c630-4581-aebb-4e88e3c9197d-kube-api-access-mrbqj\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:47.042049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.042048    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-webhook-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:47.042174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.042057    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46610332-c630-4581-aebb-4e88e3c9197d-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:19:47.583248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.583213    2563 generic.go:356] "Generic (PLEG): container finished" podID="46610332-c630-4581-aebb-4e88e3c9197d" containerID="6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de" exitCode=0
Sep 11 18:19:47.583691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.583279    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57bcb96654-bqf94"
Sep 11 18:19:47.583691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.583300    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57bcb96654-bqf94" event={"ID":"46610332-c630-4581-aebb-4e88e3c9197d","Type":"ContainerDied","Data":"6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de"}
Sep 11 18:19:47.583691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.583353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57bcb96654-bqf94" event={"ID":"46610332-c630-4581-aebb-4e88e3c9197d","Type":"ContainerDied","Data":"5f66631a128b70faf2a4abc4ebd4ae0493c7aa9039f04732d8a7938ac3f177d8"}
Sep 11 18:19:47.583691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.583380    2563 scope.go:117] "RemoveContainer" containerID="6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de"
Sep 11 18:19:47.593512 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.593491    2563 scope.go:117] "RemoveContainer" containerID="6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de"
Sep 11 18:19:47.595305 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:19:47.593822    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de\": container with ID starting with 6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de not found: ID does not exist" containerID="6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de"
Sep 11 18:19:47.595305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.593863    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de"} err="failed to get container status \"6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de\": rpc error: code = NotFound desc = could not find container \"6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de\": container with ID starting with 6737c08fa43091648c057242e71de155b60014b51c0aa7e09bc88f390c7a06de not found: ID does not exist"
Sep 11 18:19:47.610598 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.610575    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57bcb96654-bqf94"]
Sep 11 18:19:47.614638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.614618    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57bcb96654-bqf94"]
Sep 11 18:19:47.912330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:19:47.912255    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46610332-c630-4581-aebb-4e88e3c9197d" path="/var/lib/kubelet/pods/46610332-c630-4581-aebb-4e88e3c9197d/volumes"
Sep 11 18:20:05.953095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:05.953055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:06.864791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:06.864747    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:06.961211 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:06.961180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:08.389769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.389738    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:08.489600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.489561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:08.527853 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.527816    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"]
Sep 11 18:20:08.528402 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.528386    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46610332-c630-4581-aebb-4e88e3c9197d" containerName="manager"
Sep 11 18:20:08.528452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.528405    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="46610332-c630-4581-aebb-4e88e3c9197d" containerName="manager"
Sep 11 18:20:08.528507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.528496    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="46610332-c630-4581-aebb-4e88e3c9197d" containerName="manager"
Sep 11 18:20:08.537547 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.537519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.540108 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.540082    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bqwkv\""
Sep 11 18:20:08.540374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.540356    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 18:20:08.540468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.540356    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 18:20:08.541242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.541203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 18:20:08.541524 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.541470    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 18:20:08.543967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.543943    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"]
Sep 11 18:20:08.639657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.639617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87912f1f-7401-434e-9e96-24ed9db35d12-openshift-service-ca\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.639907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.639688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.639907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.639724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.639907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.639837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjx49\" (UniqueName: \"kubernetes.io/projected/87912f1f-7401-434e-9e96-24ed9db35d12-kube-api-access-wjx49\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.741058 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.741021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.741238 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.741074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.741238 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.741134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjx49\" (UniqueName: \"kubernetes.io/projected/87912f1f-7401-434e-9e96-24ed9db35d12-kube-api-access-wjx49\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.741238 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:20:08.741178    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 11 18:20:08.741238 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.741200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87912f1f-7401-434e-9e96-24ed9db35d12-openshift-service-ca\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.741460 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:20:08.741281    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 11 18:20:08.741460 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:20:08.741286    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs podName:87912f1f-7401-434e-9e96-24ed9db35d12 nodeName:}" failed. No retries permitted until 2026-09-11 18:20:09.241234847 +0000 UTC m=+577.834435399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs") pod "maas-api-56d545b69f-vg8sf" (UID: "87912f1f-7401-434e-9e96-24ed9db35d12") : secret "maas-api-metrics-tls" not found
Sep 11 18:20:08.741460 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:20:08.741403    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls podName:87912f1f-7401-434e-9e96-24ed9db35d12 nodeName:}" failed. No retries permitted until 2026-09-11 18:20:09.241369691 +0000 UTC m=+577.834570241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls") pod "maas-api-56d545b69f-vg8sf" (UID: "87912f1f-7401-434e-9e96-24ed9db35d12") : secret "maas-api-serving-cert" not found
Sep 11 18:20:08.741875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.741856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87912f1f-7401-434e-9e96-24ed9db35d12-openshift-service-ca\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.758946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.758915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjx49\" (UniqueName: \"kubernetes.io/projected/87912f1f-7401-434e-9e96-24ed9db35d12-kube-api-access-wjx49\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:08.892907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.892801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:08.996428 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:08.996392    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:09.029880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.029850    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"]
Sep 11 18:20:09.034108 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.034078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.037249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.037219    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ttngz\""
Sep 11 18:20:09.043488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.043454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 18:20:09.054255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.054224    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"]
Sep 11 18:20:09.140544 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.140511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:09.144768 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.144657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1aeb3a3-97e1-4310-a598-0346c7e1231b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-gc5jg\" (UID: \"d1aeb3a3-97e1-4310-a598-0346c7e1231b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.144918 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.144772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9677\" (UniqueName: \"kubernetes.io/projected/d1aeb3a3-97e1-4310-a598-0346c7e1231b-kube-api-access-l9677\") pod \"payload-processing-5b54b7fbb7-gc5jg\" (UID: \"d1aeb3a3-97e1-4310-a598-0346c7e1231b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.246044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.246003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1aeb3a3-97e1-4310-a598-0346c7e1231b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-gc5jg\" (UID: \"d1aeb3a3-97e1-4310-a598-0346c7e1231b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.246246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.246057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:09.246246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.246090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:09.246246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.246122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9677\" (UniqueName: \"kubernetes.io/projected/d1aeb3a3-97e1-4310-a598-0346c7e1231b-kube-api-access-l9677\") pod \"payload-processing-5b54b7fbb7-gc5jg\" (UID: \"d1aeb3a3-97e1-4310-a598-0346c7e1231b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.246972 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.246944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1aeb3a3-97e1-4310-a598-0346c7e1231b-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-gc5jg\" (UID: \"d1aeb3a3-97e1-4310-a598-0346c7e1231b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.248840 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.248812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:09.248840 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.248829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs\") pod \"maas-api-56d545b69f-vg8sf\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") " pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:09.255156 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.255115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9677\" (UniqueName: \"kubernetes.io/projected/d1aeb3a3-97e1-4310-a598-0346c7e1231b-kube-api-access-l9677\") pod \"payload-processing-5b54b7fbb7-gc5jg\" (UID: \"d1aeb3a3-97e1-4310-a598-0346c7e1231b\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.346878 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.346832    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:09.450209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.450172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:09.620884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.620852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"]
Sep 11 18:20:09.624124 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:20:09.624098    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1aeb3a3_97e1_4310_a598_0346c7e1231b.slice/crio-8666cea761bfd0300fd0ae109ee2133be9c2e64b181a6f2ea0a1f09210f14ade WatchSource:0}: Error finding container 8666cea761bfd0300fd0ae109ee2133be9c2e64b181a6f2ea0a1f09210f14ade: Status 404 returned error can't find the container with id 8666cea761bfd0300fd0ae109ee2133be9c2e64b181a6f2ea0a1f09210f14ade
Sep 11 18:20:09.690126 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.690045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg" event={"ID":"d1aeb3a3-97e1-4310-a598-0346c7e1231b","Type":"ContainerStarted","Data":"8666cea761bfd0300fd0ae109ee2133be9c2e64b181a6f2ea0a1f09210f14ade"}
Sep 11 18:20:09.729896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:09.727506    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"]
Sep 11 18:20:09.732165 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:20:09.732134    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87912f1f_7401_434e_9e96_24ed9db35d12.slice/crio-e0603309ba3b5c6b54974b7f97d8337e9fe829d6500e03dd236a05d15e112bc5 WatchSource:0}: Error finding container e0603309ba3b5c6b54974b7f97d8337e9fe829d6500e03dd236a05d15e112bc5: Status 404 returned error can't find the container with id e0603309ba3b5c6b54974b7f97d8337e9fe829d6500e03dd236a05d15e112bc5
Sep 11 18:20:10.717241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:10.717141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf" event={"ID":"87912f1f-7401-434e-9e96-24ed9db35d12","Type":"ContainerStarted","Data":"e0603309ba3b5c6b54974b7f97d8337e9fe829d6500e03dd236a05d15e112bc5"}
Sep 11 18:20:13.514942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.514808    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:13.617791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.617749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:13.745113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.745078    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg" event={"ID":"d1aeb3a3-97e1-4310-a598-0346c7e1231b","Type":"ContainerStarted","Data":"6e73329d12d8ac740755c983f8af873a8e3318e35c005a357580a8d19986e651"}
Sep 11 18:20:13.745341 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.745151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:13.746705 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.746679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf" event={"ID":"87912f1f-7401-434e-9e96-24ed9db35d12","Type":"ContainerStarted","Data":"9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729"}
Sep 11 18:20:13.746869 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.746832    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:13.766285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.766180    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg" podStartSLOduration=1.118653888 podStartE2EDuration="4.766163811s" totalImagesPullingTime="3.647509923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:20:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:20:13.764575408 +0000 UTC m=+582.357775965" watchObservedRunningTime="2026-09-11 18:20:13.766163811 +0000 UTC m=+582.359364365"
Sep 11 18:20:13.783391 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:13.783330    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf" podStartSLOduration=2.292157948 podStartE2EDuration="5.78331418s" totalImagesPullingTime="3.491156232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:20:13.783050364 +0000 UTC m=+582.376250926" watchObservedRunningTime="2026-09-11 18:20:13.78331418 +0000 UTC m=+582.376514739"
Sep 11 18:20:15.466681 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:15.466638    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:18.774637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:18.774535    2563 generic.go:356] "Generic (PLEG): container finished" podID="ed068df9-4d54-4e39-90a8-933b710e11c0" containerID="66dfe2f8585f11f5d700fcd5a5102c452473fd0bda817be0c14974406925f451" exitCode=0
Sep 11 18:20:18.774637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:18.774605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" event={"ID":"ed068df9-4d54-4e39-90a8-933b710e11c0","Type":"ContainerDied","Data":"66dfe2f8585f11f5d700fcd5a5102c452473fd0bda817be0c14974406925f451"}
Sep 11 18:20:18.775064 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:18.774998    2563 scope.go:117] "RemoveContainer" containerID="66dfe2f8585f11f5d700fcd5a5102c452473fd0bda817be0c14974406925f451"
Sep 11 18:20:18.776041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:18.776005    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:20:18.776135 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:18.776081    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:20:18.776135 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:18.776117    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:20:19.757458 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:19.757429    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:20:19.781519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:19.781488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" event={"ID":"ed068df9-4d54-4e39-90a8-933b710e11c0","Type":"ContainerStarted","Data":"218b991fd1d1a454863b834714203d8bd87e988cfab293559e8fda3f60435034"}
Sep 11 18:20:20.234727 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:20.234679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:20:20.234962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:20.234939    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-2gjdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 18:20:20.235023 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:20.235004    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" podUID="ed068df9-4d54-4e39-90a8-933b710e11c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 18:20:20.735028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:20.734999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:20.837752 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:20.837718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:21.234506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:21.234468    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-2gjdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 18:20:21.234683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:21.234540    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" podUID="ed068df9-4d54-4e39-90a8-933b710e11c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 18:20:22.235205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:22.235163    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-2gjdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 18:20:22.235640 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:22.235222    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" podUID="ed068df9-4d54-4e39-90a8-933b710e11c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 18:20:23.234725 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:23.234685    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-2gjdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 18:20:23.234900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:23.234749    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" podUID="ed068df9-4d54-4e39-90a8-933b710e11c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 18:20:24.234902 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:24.234851    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-2gjdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 18:20:24.235342 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:24.234934    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw" podUID="ed068df9-4d54-4e39-90a8-933b710e11c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 18:20:24.754061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:24.754028    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-gc5jg"
Sep 11 18:20:25.238318 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:25.238252    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:20:25.238781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:25.238763    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:20:25.239490 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:25.239469    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2gjdw"
Sep 11 18:20:26.245568 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:26.245539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:26.346892 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:26.346858    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:31.850944 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:31.850911    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:20:31.852735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:31.852715    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:20:32.893926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:32.893894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:33.528474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:33.528432    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:44.794937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:44.794899    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:47.493462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.493427    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"]
Sep 11 18:20:47.498893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.498857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.502111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.502082    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 18:20:47.502349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.502104    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 18:20:47.502500 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.502105    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qdfkl\""
Sep 11 18:20:47.502500 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.502148    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 18:20:47.507969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.507942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"]
Sep 11 18:20:47.521392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.521354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:20:47.665049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.665008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.665049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.665056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.665297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.665075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.665297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.665159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.665297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.665196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/703963e3-d628-4e2d-8521-3bcde98f6020-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.665297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.665234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdnk5\" (UniqueName: \"kubernetes.io/projected/703963e3-d628-4e2d-8521-3bcde98f6020-kube-api-access-mdnk5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766376 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766376 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/703963e3-d628-4e2d-8521-3bcde98f6020-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766376 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdnk5\" (UniqueName: \"kubernetes.io/projected/703963e3-d628-4e2d-8521-3bcde98f6020-kube-api-access-mdnk5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.766880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.766825    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.768770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.768739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/703963e3-d628-4e2d-8521-3bcde98f6020-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.768974 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.768958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/703963e3-d628-4e2d-8521-3bcde98f6020-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.775253 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.775225    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdnk5\" (UniqueName: \"kubernetes.io/projected/703963e3-d628-4e2d-8521-3bcde98f6020-kube-api-access-mdnk5\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gr95g\" (UID: \"703963e3-d628-4e2d-8521-3bcde98f6020\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:47.811894 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:47.811852    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:48.100405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:48.100377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"]
Sep 11 18:20:48.103003 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:20:48.102970    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703963e3_d628_4e2d_8521_3bcde98f6020.slice/crio-d00ad1d28fdbcf2d62c32eec3c9f5996704642334b306d9d77e7d4a4ea3ae935 WatchSource:0}: Error finding container d00ad1d28fdbcf2d62c32eec3c9f5996704642334b306d9d77e7d4a4ea3ae935: Status 404 returned error can't find the container with id d00ad1d28fdbcf2d62c32eec3c9f5996704642334b306d9d77e7d4a4ea3ae935
Sep 11 18:20:48.104668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:48.104642    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:20:48.926131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:48.926090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g" event={"ID":"703963e3-d628-4e2d-8521-3bcde98f6020","Type":"ContainerStarted","Data":"d00ad1d28fdbcf2d62c32eec3c9f5996704642334b306d9d77e7d4a4ea3ae935"}
Sep 11 18:20:49.932303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:49.932242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g" event={"ID":"703963e3-d628-4e2d-8521-3bcde98f6020","Type":"ContainerStarted","Data":"55b6a2b42eb52e4eb08a720c72d5e0d9282f158587a07c19e3d6fdba0ef3dd40"}
Sep 11 18:20:49.952718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:49.952649    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g" podStartSLOduration=1.862584705 podStartE2EDuration="2.95262433s" totalImagesPullingTime="1.090039625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:20:49.952292681 +0000 UTC m=+618.545493239" watchObservedRunningTime="2026-09-11 18:20:49.95262433 +0000 UTC m=+618.545824888"
Sep 11 18:20:57.813170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:57.813069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:57.813170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:57.813144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:57.825888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:57.825859    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:20:57.981595 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:20:57.981563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gr95g"
Sep 11 18:21:05.217734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:05.217668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:21:16.437793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:16.437763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:21:27.893417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:27.893384    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"]
Sep 11 18:21:27.897147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:27.897121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:27.899476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:27.899450    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 18:21:27.913016 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:27.912990    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"]
Sep 11 18:21:27.915926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:27.915894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:21:28.040546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.040501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.040546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.040542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.040767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.040566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5pl7\" (UniqueName: \"kubernetes.io/projected/c5e5f941-2b3a-4764-a35c-b68d89773a76-kube-api-access-z5pl7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.040767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.040609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.040767 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.040668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.040871 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.040787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5e5f941-2b3a-4764-a35c-b68d89773a76-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5e5f941-2b3a-4764-a35c-b68d89773a76-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5pl7\" (UniqueName: \"kubernetes.io/projected/c5e5f941-2b3a-4764-a35c-b68d89773a76-kube-api-access-z5pl7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142645 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.142887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.142856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.144729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.144664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5e5f941-2b3a-4764-a35c-b68d89773a76-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.144985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.144966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5e5f941-2b3a-4764-a35c-b68d89773a76-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.150854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.150832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5pl7\" (UniqueName: \"kubernetes.io/projected/c5e5f941-2b3a-4764-a35c-b68d89773a76-kube-api-access-z5pl7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6\" (UID: \"c5e5f941-2b3a-4764-a35c-b68d89773a76\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.208470 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.208427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:28.481474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:28.481444    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"]
Sep 11 18:21:28.483225 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:21:28.483196    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e5f941_2b3a_4764_a35c_b68d89773a76.slice/crio-623611afde8128a6f67dc89e66ae2d60cbd67e4435dd76e857d744aed421de0f WatchSource:0}: Error finding container 623611afde8128a6f67dc89e66ae2d60cbd67e4435dd76e857d744aed421de0f: Status 404 returned error can't find the container with id 623611afde8128a6f67dc89e66ae2d60cbd67e4435dd76e857d744aed421de0f
Sep 11 18:21:29.119452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:29.119416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6" event={"ID":"c5e5f941-2b3a-4764-a35c-b68d89773a76","Type":"ContainerStarted","Data":"449bb07f633c397d74e8a8e33bceb61ffb547082355b00f0422ee2743b20bfaa"}
Sep 11 18:21:29.119860 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:29.119470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6" event={"ID":"c5e5f941-2b3a-4764-a35c-b68d89773a76","Type":"ContainerStarted","Data":"623611afde8128a6f67dc89e66ae2d60cbd67e4435dd76e857d744aed421de0f"}
Sep 11 18:21:29.151801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:29.151732    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6" podStartSLOduration=1.933689311 podStartE2EDuration="2.151707231s" totalImagesPullingTime="218.01792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:21:29.150538432 +0000 UTC m=+657.743738989" watchObservedRunningTime="2026-09-11 18:21:29.151707231 +0000 UTC m=+657.744907793"
Sep 11 18:21:38.208922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:38.208883    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:38.209400 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:38.208955    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:38.221956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:38.221928    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:39.184597 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:39.184558    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6"
Sep 11 18:21:40.115952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:40.115909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:21:49.599543 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.599509    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"]
Sep 11 18:21:49.603493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.603469    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.605716 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.605689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 18:21:49.613170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.613136    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"]
Sep 11 18:21:49.621632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.621596    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:21:49.646360 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.646327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.646520 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.646378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.646520 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.646430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/857309fa-3fa4-4717-8c29-6e69febe291c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.646520 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.646491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvz2\" (UniqueName: \"kubernetes.io/projected/857309fa-3fa4-4717-8c29-6e69febe291c-kube-api-access-zpvz2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.646657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.646551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.646657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.646583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/857309fa-3fa4-4717-8c29-6e69febe291c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747716 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvz2\" (UniqueName: \"kubernetes.io/projected/857309fa-3fa4-4717-8c29-6e69febe291c-kube-api-access-zpvz2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747882 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.747882 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.747811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.748049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.748030    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.749982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.749950    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/857309fa-3fa4-4717-8c29-6e69febe291c-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.750185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.750169    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/857309fa-3fa4-4717-8c29-6e69febe291c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.756704 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.756680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvz2\" (UniqueName: \"kubernetes.io/projected/857309fa-3fa4-4717-8c29-6e69febe291c-kube-api-access-zpvz2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p\" (UID: \"857309fa-3fa4-4717-8c29-6e69febe291c\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:49.913406 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:49.913324    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:50.187030 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:50.187004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"]
Sep 11 18:21:50.189130 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:21:50.189083    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod857309fa_3fa4_4717_8c29_6e69febe291c.slice/crio-aded999686ec42c274b5825e74829b5f78a6f51a85686378da02ad11e6b18c70 WatchSource:0}: Error finding container aded999686ec42c274b5825e74829b5f78a6f51a85686378da02ad11e6b18c70: Status 404 returned error can't find the container with id aded999686ec42c274b5825e74829b5f78a6f51a85686378da02ad11e6b18c70
Sep 11 18:21:50.221330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:50.221291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p" event={"ID":"857309fa-3fa4-4717-8c29-6e69febe291c","Type":"ContainerStarted","Data":"aded999686ec42c274b5825e74829b5f78a6f51a85686378da02ad11e6b18c70"}
Sep 11 18:21:51.227999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:51.227963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p" event={"ID":"857309fa-3fa4-4717-8c29-6e69febe291c","Type":"ContainerStarted","Data":"f47e8b6279724f051933a5782cbcc38ac5acd365648c9c4b9debd3ebdd55253f"}
Sep 11 18:21:51.247557 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:51.247506    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p" podStartSLOduration=2.061932413 podStartE2EDuration="2.247464642s" totalImagesPullingTime="185.532229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:21:51.245397615 +0000 UTC m=+679.838598173" watchObservedRunningTime="2026-09-11 18:21:51.247464642 +0000 UTC m=+679.840665216"
Sep 11 18:21:59.914166 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:59.914128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:59.914602 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:59.914177    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:21:59.926777 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:21:59.926749    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:22:00.280444 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:22:00.280411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p"
Sep 11 18:22:23.901851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:22:23.901806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:22:28.785412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:22:28.785374    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:22:39.726242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:22:39.726208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:22:42.743466 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:22:42.743431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:04.410689 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.410650    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9576b5555-k69zb"]
Sep 11 18:23:04.414084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.414065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.423119 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.423094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9576b5555-k69zb"]
Sep 11 18:23:04.466354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.466320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5c5d0def-fccd-40ec-ac33-31898ff4f6bf-tls-cert\") pod \"authorino-9576b5555-k69zb\" (UID: \"5c5d0def-fccd-40ec-ac33-31898ff4f6bf\") " pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.466529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.466363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btx26\" (UniqueName: \"kubernetes.io/projected/5c5d0def-fccd-40ec-ac33-31898ff4f6bf-kube-api-access-btx26\") pod \"authorino-9576b5555-k69zb\" (UID: \"5c5d0def-fccd-40ec-ac33-31898ff4f6bf\") " pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.567149 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.567108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5c5d0def-fccd-40ec-ac33-31898ff4f6bf-tls-cert\") pod \"authorino-9576b5555-k69zb\" (UID: \"5c5d0def-fccd-40ec-ac33-31898ff4f6bf\") " pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.567149 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.567149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btx26\" (UniqueName: \"kubernetes.io/projected/5c5d0def-fccd-40ec-ac33-31898ff4f6bf-kube-api-access-btx26\") pod \"authorino-9576b5555-k69zb\" (UID: \"5c5d0def-fccd-40ec-ac33-31898ff4f6bf\") " pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.569697 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.569663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5c5d0def-fccd-40ec-ac33-31898ff4f6bf-tls-cert\") pod \"authorino-9576b5555-k69zb\" (UID: \"5c5d0def-fccd-40ec-ac33-31898ff4f6bf\") " pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.575375 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.575350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btx26\" (UniqueName: \"kubernetes.io/projected/5c5d0def-fccd-40ec-ac33-31898ff4f6bf-kube-api-access-btx26\") pod \"authorino-9576b5555-k69zb\" (UID: \"5c5d0def-fccd-40ec-ac33-31898ff4f6bf\") " pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.724095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.724006    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9576b5555-k69zb"
Sep 11 18:23:04.997302 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:04.997281    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9576b5555-k69zb"]
Sep 11 18:23:05.000315 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:05.000286    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c5d0def_fccd_40ec_ac33_31898ff4f6bf.slice/crio-3b4743d51a3b2a15b1cce338572f5fe68e716452456ab20468c5d8a93be530db WatchSource:0}: Error finding container 3b4743d51a3b2a15b1cce338572f5fe68e716452456ab20468c5d8a93be530db: Status 404 returned error can't find the container with id 3b4743d51a3b2a15b1cce338572f5fe68e716452456ab20468c5d8a93be530db
Sep 11 18:23:05.598170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:05.598129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9576b5555-k69zb" event={"ID":"5c5d0def-fccd-40ec-ac33-31898ff4f6bf","Type":"ContainerStarted","Data":"3b4743d51a3b2a15b1cce338572f5fe68e716452456ab20468c5d8a93be530db"}
Sep 11 18:23:06.606370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.606331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9576b5555-k69zb" event={"ID":"5c5d0def-fccd-40ec-ac33-31898ff4f6bf","Type":"ContainerStarted","Data":"597d0317d2eb3f09391958585dbd1b3cc1707dd104d76a6c1382b3eeeaaa067b"}
Sep 11 18:23:06.623665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.623608    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9576b5555-k69zb" podStartSLOduration=2.038263191 podStartE2EDuration="2.623586897s" totalImagesPullingTime="585.323706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:06.622549721 +0000 UTC m=+755.215750279" watchObservedRunningTime="2026-09-11 18:23:06.623586897 +0000 UTC m=+755.216787455"
Sep 11 18:23:06.658119 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.658073    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76b96f6658-7gsx6"]
Sep 11 18:23:06.658339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.658312    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-76b96f6658-7gsx6" podUID="d3187c01-70bb-4717-a092-62a0873474ff" containerName="authorino" containerID="cri-o://439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79" gracePeriod=30
Sep 11 18:23:06.916041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.916009    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:23:06.991503 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.991465    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3187c01-70bb-4717-a092-62a0873474ff-tls-cert\") pod \"d3187c01-70bb-4717-a092-62a0873474ff\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") "
Sep 11 18:23:06.991660 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.991625    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86fhl\" (UniqueName: \"kubernetes.io/projected/d3187c01-70bb-4717-a092-62a0873474ff-kube-api-access-86fhl\") pod \"d3187c01-70bb-4717-a092-62a0873474ff\" (UID: \"d3187c01-70bb-4717-a092-62a0873474ff\") "
Sep 11 18:23:06.993663 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:06.993631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3187c01-70bb-4717-a092-62a0873474ff-kube-api-access-86fhl" (OuterVolumeSpecName: "kube-api-access-86fhl") pod "d3187c01-70bb-4717-a092-62a0873474ff" (UID: "d3187c01-70bb-4717-a092-62a0873474ff"). InnerVolumeSpecName "kube-api-access-86fhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:23:07.001806 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.001777    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3187c01-70bb-4717-a092-62a0873474ff-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d3187c01-70bb-4717-a092-62a0873474ff" (UID: "d3187c01-70bb-4717-a092-62a0873474ff"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:23:07.093157 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.093120    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d3187c01-70bb-4717-a092-62a0873474ff-tls-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:23:07.093157 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.093153    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86fhl\" (UniqueName: \"kubernetes.io/projected/d3187c01-70bb-4717-a092-62a0873474ff-kube-api-access-86fhl\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:23:07.159584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.159510    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:07.262829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.262759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:07.613853 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.613816    2563 generic.go:356] "Generic (PLEG): container finished" podID="d3187c01-70bb-4717-a092-62a0873474ff" containerID="439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79" exitCode=0
Sep 11 18:23:07.614281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.613888    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b96f6658-7gsx6"
Sep 11 18:23:07.614281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.613908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b96f6658-7gsx6" event={"ID":"d3187c01-70bb-4717-a092-62a0873474ff","Type":"ContainerDied","Data":"439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79"}
Sep 11 18:23:07.614281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.613953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b96f6658-7gsx6" event={"ID":"d3187c01-70bb-4717-a092-62a0873474ff","Type":"ContainerDied","Data":"eddebd3b6396fea91141b984530e57ea7ce778e76701c01cdfa11bdc0ec41970"}
Sep 11 18:23:07.614281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.613971    2563 scope.go:117] "RemoveContainer" containerID="439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79"
Sep 11 18:23:07.622719 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.622703    2563 scope.go:117] "RemoveContainer" containerID="439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79"
Sep 11 18:23:07.622971 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:23:07.622953    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79\": container with ID starting with 439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79 not found: ID does not exist" containerID="439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79"
Sep 11 18:23:07.623051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.622981    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79"} err="failed to get container status \"439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79\": rpc error: code = NotFound desc = could not find container \"439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79\": container with ID starting with 439d71eccadce52c976e56789f154bf0ff7baab4fe0db2b6413ce13fd3196a79 not found: ID does not exist"
Sep 11 18:23:07.637390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.637369    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76b96f6658-7gsx6"]
Sep 11 18:23:07.646020 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.646000    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76b96f6658-7gsx6"]
Sep 11 18:23:07.912946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:07.912854    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3187c01-70bb-4717-a092-62a0873474ff" path="/var/lib/kubelet/pods/d3187c01-70bb-4717-a092-62a0873474ff/volumes"
Sep 11 18:23:08.271068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:08.270972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:08.370474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:08.370431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:23.664808 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:23.664766    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:23.766249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:23.766201    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:23.866352 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:23.866314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.027679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.027635    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.129398 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.123255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.140614 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.140581    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"]
Sep 11 18:23:24.141582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.141557    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3187c01-70bb-4717-a092-62a0873474ff" containerName="authorino"
Sep 11 18:23:24.141582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.141581    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3187c01-70bb-4717-a092-62a0873474ff" containerName="authorino"
Sep 11 18:23:24.141758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.141750    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3187c01-70bb-4717-a092-62a0873474ff" containerName="authorino"
Sep 11 18:23:24.161281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.160909    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.168405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.168375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"]
Sep 11 18:23:24.168613 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.168592    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f63893-openshift-default-dockercfg-cd75m\""
Sep 11 18:23:24.222863 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.222826    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.271967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.271932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-data\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.271990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-podinfo\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-socket\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9skc\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-kube-api-access-x9skc\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-certs\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-token\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272203    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-credential-socket\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-envoy\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.272374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.272334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istiod-ca-cert\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.372960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istiod-ca-cert\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-data\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373051    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-podinfo\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-socket\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9skc\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-kube-api-access-x9skc\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-certs\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-token\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373530 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-credential-socket\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373530 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-envoy\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.373980 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.373891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-certs\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.374619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.374128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-credential-socket\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.374619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.374333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-data\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.374619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.374509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-socket\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.374823 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.374677    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istiod-ca-cert\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.376818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.376777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-envoy\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.377348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.377324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-podinfo\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.382097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.382072    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-token\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.382971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.382942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9skc\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-kube-api-access-x9skc\") pod \"e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") " pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.480501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.480470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:24.490418 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.490387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.582395 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.582367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.710004 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.709928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:24.781058 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.781028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"]
Sep 11 18:23:24.784347 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:24.784321    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98fe6a7_7c7b_45c8_b5c7_18e7a3057067.slice/crio-0b9e28d8dd41017255fbadbf074b6a03074b5781edb5e0a30cacf1185bb1b8cc WatchSource:0}: Error finding container 0b9e28d8dd41017255fbadbf074b6a03074b5781edb5e0a30cacf1185bb1b8cc: Status 404 returned error can't find the container with id 0b9e28d8dd41017255fbadbf074b6a03074b5781edb5e0a30cacf1185bb1b8cc
Sep 11 18:23:24.786441 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.786393    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:23:24.786554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.786486    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:23:24.786554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:24.786533    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:23:25.107703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:25.107667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:25.209096 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:25.209061    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:25.521646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:25.521571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:25.710124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:25.710090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" event={"ID":"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067","Type":"ContainerStarted","Data":"0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"}
Sep 11 18:23:25.710486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:25.710152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" event={"ID":"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067","Type":"ContainerStarted","Data":"0b9e28d8dd41017255fbadbf074b6a03074b5781edb5e0a30cacf1185bb1b8cc"}
Sep 11 18:23:25.731726 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:25.731683    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podStartSLOduration=1.731668507 podStartE2EDuration="1.731668507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:25.729827934 +0000 UTC m=+774.323028490" watchObservedRunningTime="2026-09-11 18:23:25.731668507 +0000 UTC m=+774.324869064"
Sep 11 18:23:26.481412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:26.481377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:26.486459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:26.486434    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:26.714847 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:26.714815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:26.715670 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:26.715651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:27.011967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:27.011930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:27.119130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:27.119100    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:27.214655 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:27.214625    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:27.637435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:27.637396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:27.791453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:27.791409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:27.880988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:27.880956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:28.505932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.505897    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"]
Sep 11 18:23:28.509416 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.509395    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.511979 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.511957    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-d73031-serving-cert\""
Sep 11 18:23:28.516248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.516121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.516248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.516177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8jps\" (UniqueName: \"kubernetes.io/projected/15df66de-fd57-4bbd-873d-6e759ec3f367-kube-api-access-h8jps\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.516248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.516208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15df66de-fd57-4bbd-873d-6e759ec3f367-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.516475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.516328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-metrics-certs\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.519067 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.519046    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"]
Sep 11 18:23:28.565285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.565232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:28.617120 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.617097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.617239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.617133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8jps\" (UniqueName: \"kubernetes.io/projected/15df66de-fd57-4bbd-873d-6e759ec3f367-kube-api-access-h8jps\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.617239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.617161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15df66de-fd57-4bbd-873d-6e759ec3f367-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.617350 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:23:28.617234    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-serving-cert: secret "maas-api-e2e-worker-w0-d73031-serving-cert" not found
Sep 11 18:23:28.617350 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.617330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-metrics-certs\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.617350 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:23:28.617337    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls podName:15df66de-fd57-4bbd-873d-6e759ec3f367 nodeName:}" failed. No retries permitted until 2026-09-11 18:23:29.117317604 +0000 UTC m=+777.710518155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls") pod "maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" (UID: "15df66de-fd57-4bbd-873d-6e759ec3f367") : secret "maas-api-e2e-worker-w0-d73031-serving-cert" not found
Sep 11 18:23:28.617793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.617765    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15df66de-fd57-4bbd-873d-6e759ec3f367-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.619883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.619861    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-metrics-certs\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.626501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.626480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8jps\" (UniqueName: \"kubernetes.io/projected/15df66de-fd57-4bbd-873d-6e759ec3f367-kube-api-access-h8jps\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:28.667254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.667228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:28.767860 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:28.767785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:29.053516 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.053489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:29.122145 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.122109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:29.124880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.124852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls\") pod \"maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:29.160372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.160341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:29.188512 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.188477    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"]
Sep 11 18:23:29.192667 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.192644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.195037 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.195017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-d73031\""
Sep 11 18:23:29.195133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.195021    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-d73031-dockercfg-plwbn\""
Sep 11 18:23:29.203748 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.203726    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"]
Sep 11 18:23:29.222683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.222656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sshw\" (UniqueName: \"kubernetes.io/projected/0756087c-fe35-4c07-b3de-961ead072e49-kube-api-access-2sshw\") pod \"payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.222796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.222746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0756087c-fe35-4c07-b3de-961ead072e49-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.323594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.323510    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sshw\" (UniqueName: \"kubernetes.io/projected/0756087c-fe35-4c07-b3de-961ead072e49-kube-api-access-2sshw\") pod \"payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.323594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.323563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0756087c-fe35-4c07-b3de-961ead072e49-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.324361 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.324342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0756087c-fe35-4c07-b3de-961ead072e49-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.334741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.334712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sshw\" (UniqueName: \"kubernetes.io/projected/0756087c-fe35-4c07-b3de-961ead072e49-kube-api-access-2sshw\") pod \"payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.421968 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.421930    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:29.504968 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.504814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:29.614626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.614541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:29.693508 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.693485    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"]
Sep 11 18:23:29.695001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.694964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:29.695419 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:29.695393    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15df66de_fd57_4bbd_873d_6e759ec3f367.slice/crio-7012268aa14700aef75921bc96e0873f805cf8d658e7eed863e4ff2175f25110 WatchSource:0}: Error finding container 7012268aa14700aef75921bc96e0873f805cf8d658e7eed863e4ff2175f25110: Status 404 returned error can't find the container with id 7012268aa14700aef75921bc96e0873f805cf8d658e7eed863e4ff2175f25110
Sep 11 18:23:29.733309 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.733284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" event={"ID":"15df66de-fd57-4bbd-873d-6e759ec3f367","Type":"ContainerStarted","Data":"7012268aa14700aef75921bc96e0873f805cf8d658e7eed863e4ff2175f25110"}
Sep 11 18:23:29.776008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.770645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:29.783405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:29.783384    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"]
Sep 11 18:23:29.785015 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:29.784990    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0756087c_fe35_4c07_b3de_961ead072e49.slice/crio-ed4d9582f7df7988190fec403c0811510a9644a8035a40bb0d7e14ed7e394a7b WatchSource:0}: Error finding container ed4d9582f7df7988190fec403c0811510a9644a8035a40bb0d7e14ed7e394a7b: Status 404 returned error can't find the container with id ed4d9582f7df7988190fec403c0811510a9644a8035a40bb0d7e14ed7e394a7b
Sep 11 18:23:30.212315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.212290    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:30.312239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.312200    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:30.343541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.343514    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"]
Sep 11 18:23:30.347830 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.347804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.350150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.350094    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-553596-dockercfg-v2rdr\""
Sep 11 18:23:30.350150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.350120    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-553596\""
Sep 11 18:23:30.360124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.359782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"]
Sep 11 18:23:30.434064 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.433993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feac80ec-5627-4b6e-8e32-6d9435693ae0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.434198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.434146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qdv9\" (UniqueName: \"kubernetes.io/projected/feac80ec-5627-4b6e-8e32-6d9435693ae0-kube-api-access-4qdv9\") pod \"payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.535464 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.535425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qdv9\" (UniqueName: \"kubernetes.io/projected/feac80ec-5627-4b6e-8e32-6d9435693ae0-kube-api-access-4qdv9\") pod \"payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.535638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.535497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feac80ec-5627-4b6e-8e32-6d9435693ae0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.536208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.536182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feac80ec-5627-4b6e-8e32-6d9435693ae0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.543785 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.543762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qdv9\" (UniqueName: \"kubernetes.io/projected/feac80ec-5627-4b6e-8e32-6d9435693ae0-kube-api-access-4qdv9\") pod \"payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.662237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.662208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:30.677233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.677212    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:30.742130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.742087    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh" event={"ID":"0756087c-fe35-4c07-b3de-961ead072e49","Type":"ContainerStarted","Data":"97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857"}
Sep 11 18:23:30.742322 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.742249    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:30.742322 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.742292    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh" event={"ID":"0756087c-fe35-4c07-b3de-961ead072e49","Type":"ContainerStarted","Data":"ed4d9582f7df7988190fec403c0811510a9644a8035a40bb0d7e14ed7e394a7b"}
Sep 11 18:23:30.744125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.744092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" event={"ID":"15df66de-fd57-4bbd-873d-6e759ec3f367","Type":"ContainerStarted","Data":"e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae"}
Sep 11 18:23:30.744289 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.744199    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:30.762141 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.762087    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh" podStartSLOduration=1.762064102 podStartE2EDuration="1.762064102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:30.759194029 +0000 UTC m=+779.352394584" watchObservedRunningTime="2026-09-11 18:23:30.762064102 +0000 UTC m=+779.355264664"
Sep 11 18:23:30.778364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.778319    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" podStartSLOduration=2.360992576 podStartE2EDuration="2.778303173s" totalImagesPullingTime="417.310597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:30.776100779 +0000 UTC m=+779.369301335" watchObservedRunningTime="2026-09-11 18:23:30.778303173 +0000 UTC m=+779.371503729"
Sep 11 18:23:30.937648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:30.937626    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"]
Sep 11 18:23:30.939909 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:30.939881    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeac80ec_5627_4b6e_8e32_6d9435693ae0.slice/crio-453d354f9b5e78c7776d53c5594bd3bd65609c83a27a5385b820ab3958b8357e WatchSource:0}: Error finding container 453d354f9b5e78c7776d53c5594bd3bd65609c83a27a5385b820ab3958b8357e: Status 404 returned error can't find the container with id 453d354f9b5e78c7776d53c5594bd3bd65609c83a27a5385b820ab3958b8357e
Sep 11 18:23:31.191008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.190976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:31.528742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.528708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:31.622340 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.622305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:31.722645 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.722611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:31.752880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.752839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr" event={"ID":"feac80ec-5627-4b6e-8e32-6d9435693ae0","Type":"ContainerStarted","Data":"cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d"}
Sep 11 18:23:31.753044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.752900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr" event={"ID":"feac80ec-5627-4b6e-8e32-6d9435693ae0","Type":"ContainerStarted","Data":"453d354f9b5e78c7776d53c5594bd3bd65609c83a27a5385b820ab3958b8357e"}
Sep 11 18:23:31.753106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.753060    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:31.771284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:31.771228    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr" podStartSLOduration=1.771214262 podStartE2EDuration="1.771214262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:31.769397024 +0000 UTC m=+780.362597581" watchObservedRunningTime="2026-09-11 18:23:31.771214262 +0000 UTC m=+780.364414819"
Sep 11 18:23:32.002622 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.002592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:32.104388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.104358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:32.105743 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.105725    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"]
Sep 11 18:23:32.110824 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.110805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.118463 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.113852    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-f63893\""
Sep 11 18:23:32.121187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.121165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"]
Sep 11 18:23:32.150911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.150881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vg87\" (UniqueName: \"kubernetes.io/projected/2e43710c-a78c-461f-808e-49951e027dd7-kube-api-access-9vg87\") pod \"payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.151043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.150994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e43710c-a78c-461f-808e-49951e027dd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.252126 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.252037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vg87\" (UniqueName: \"kubernetes.io/projected/2e43710c-a78c-461f-808e-49951e027dd7-kube-api-access-9vg87\") pod \"payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.252310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.252180    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e43710c-a78c-461f-808e-49951e027dd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.253110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.253084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e43710c-a78c-461f-808e-49951e027dd7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.260091 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.260034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vg87\" (UniqueName: \"kubernetes.io/projected/2e43710c-a78c-461f-808e-49951e027dd7-kube-api-access-9vg87\") pod \"payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.399061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.399030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:32.430829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.430807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:32.558328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.558301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:32.659131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.659073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:32.702064 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.702041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"]
Sep 11 18:23:32.703918 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:32.703882    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e43710c_a78c_461f_808e_49951e027dd7.slice/crio-a24eb9e24e2f743b4b1dee4c75c5a0ee84f025ccc17ec5d1b90c16531425c1dc WatchSource:0}: Error finding container a24eb9e24e2f743b4b1dee4c75c5a0ee84f025ccc17ec5d1b90c16531425c1dc: Status 404 returned error can't find the container with id a24eb9e24e2f743b4b1dee4c75c5a0ee84f025ccc17ec5d1b90c16531425c1dc
Sep 11 18:23:32.758638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.758611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp" event={"ID":"2e43710c-a78c-461f-808e-49951e027dd7","Type":"ContainerStarted","Data":"a24eb9e24e2f743b4b1dee4c75c5a0ee84f025ccc17ec5d1b90c16531425c1dc"}
Sep 11 18:23:32.772000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:32.771976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:33.057891 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:33.057848    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:33.178335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:33.178303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:33.474758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:33.474680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:33.766459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:33.766372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp" event={"ID":"2e43710c-a78c-461f-808e-49951e027dd7","Type":"ContainerStarted","Data":"99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c"}
Sep 11 18:23:33.766886 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:33.766492    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:33.788694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:33.788643    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp" podStartSLOduration=1.788625799 podStartE2EDuration="1.788625799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:33.786430712 +0000 UTC m=+782.379631271" watchObservedRunningTime="2026-09-11 18:23:33.788625799 +0000 UTC m=+782.381826356"
Sep 11 18:23:34.005593 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:34.005563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:35.241907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:35.241866    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:35.353134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:35.353105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:35.817693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:35.817657    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:35.948483 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:35.948435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:36.602402 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:36.602363    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:36.759358 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:36.759319    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:23:37.508062 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:37.507997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:38.397925 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.397878    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"]
Sep 11 18:23:38.430797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.430762    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:38.438379 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.437481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"]
Sep 11 18:23:38.438379 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.437622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.441183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.441151    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2f7a9131-serving-cert\""
Sep 11 18:23:38.524928 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.524886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzhl\" (UniqueName: \"kubernetes.io/projected/c7f8341e-c86d-41ed-9d71-5aee54d051d2-kube-api-access-bjzhl\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.526287 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.526237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-metrics-certs\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.526637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.526617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f8341e-c86d-41ed-9d71-5aee54d051d2-openshift-service-ca\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.526802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.526785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.529905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.529876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:38.628439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.628398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f8341e-c86d-41ed-9d71-5aee54d051d2-openshift-service-ca\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.628439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.628443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.628811 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.628508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzhl\" (UniqueName: \"kubernetes.io/projected/c7f8341e-c86d-41ed-9d71-5aee54d051d2-kube-api-access-bjzhl\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.628811 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.628608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-metrics-certs\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.628811 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:23:38.628638    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-serving-cert: secret "maas-api-e2e-ait-2f7a9131-serving-cert" not found
Sep 11 18:23:38.628811 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:23:38.628713    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls podName:c7f8341e-c86d-41ed-9d71-5aee54d051d2 nodeName:}" failed. No retries permitted until 2026-09-11 18:23:39.128691549 +0000 UTC m=+787.721892084 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls") pod "maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" (UID: "c7f8341e-c86d-41ed-9d71-5aee54d051d2") : secret "maas-api-e2e-ait-2f7a9131-serving-cert" not found
Sep 11 18:23:38.629317 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.629295    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f8341e-c86d-41ed-9d71-5aee54d051d2-openshift-service-ca\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.631273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.631233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-metrics-certs\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.639635 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.639607    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzhl\" (UniqueName: \"kubernetes.io/projected/c7f8341e-c86d-41ed-9d71-5aee54d051d2-kube-api-access-bjzhl\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:38.719087 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:38.718979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:39.048110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.048076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:39.134552 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.134518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:39.136977 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.136944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls\") pod \"maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:39.148377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.148343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:39.338588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.338501    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"]
Sep 11 18:23:39.342340 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.342319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.345037 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.344896    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2f7a9131-dockercfg-6lc8l\""
Sep 11 18:23:39.345249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.345219    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2f7a9131\""
Sep 11 18:23:39.350595 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.350562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:39.351320 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.351293    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"]
Sep 11 18:23:39.438128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.438080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g9fp\" (UniqueName: \"kubernetes.io/projected/17abdb1f-ccf5-41dc-86b1-fa5115caf095-kube-api-access-2g9fp\") pod \"payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") " pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.438584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.438186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17abdb1f-ccf5-41dc-86b1-fa5115caf095-plugins-config-volume\") pod \"payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") " pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.539445 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.539407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17abdb1f-ccf5-41dc-86b1-fa5115caf095-plugins-config-volume\") pod \"payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") " pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.539635 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.539520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g9fp\" (UniqueName: \"kubernetes.io/projected/17abdb1f-ccf5-41dc-86b1-fa5115caf095-kube-api-access-2g9fp\") pod \"payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") " pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.540200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.540178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17abdb1f-ccf5-41dc-86b1-fa5115caf095-plugins-config-volume\") pod \"payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") " pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.547479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.547456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g9fp\" (UniqueName: \"kubernetes.io/projected/17abdb1f-ccf5-41dc-86b1-fa5115caf095-kube-api-access-2g9fp\") pod \"payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") " pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.620349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.618410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:39.645883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.645854    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"]
Sep 11 18:23:39.648041 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:39.648010    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f8341e_c86d_41ed_9d71_5aee54d051d2.slice/crio-2c9b164d34ac1dad2ed31beb93456ba54b8693ff772f50b44234fb73827fa739 WatchSource:0}: Error finding container 2c9b164d34ac1dad2ed31beb93456ba54b8693ff772f50b44234fb73827fa739: Status 404 returned error can't find the container with id 2c9b164d34ac1dad2ed31beb93456ba54b8693ff772f50b44234fb73827fa739
Sep 11 18:23:39.658249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.658222    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:39.829797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.829765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" event={"ID":"c7f8341e-c86d-41ed-9d71-5aee54d051d2","Type":"ContainerStarted","Data":"2c9b164d34ac1dad2ed31beb93456ba54b8693ff772f50b44234fb73827fa739"}
Sep 11 18:23:39.961154 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:39.961131    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"]
Sep 11 18:23:39.963486 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:39.963446    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17abdb1f_ccf5_41dc_86b1_fa5115caf095.slice/crio-4beb71ac4657ab4c6e1cbcc28e7cd5699963678e97eaa9f863128d129add3067 WatchSource:0}: Error finding container 4beb71ac4657ab4c6e1cbcc28e7cd5699963678e97eaa9f863128d129add3067: Status 404 returned error can't find the container with id 4beb71ac4657ab4c6e1cbcc28e7cd5699963678e97eaa9f863128d129add3067
Sep 11 18:23:40.599854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.599815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:40.682510 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.682475    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:40.837926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.837887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" event={"ID":"c7f8341e-c86d-41ed-9d71-5aee54d051d2","Type":"ContainerStarted","Data":"24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5"}
Sep 11 18:23:40.838132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.838085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:40.839739 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.839710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn" event={"ID":"17abdb1f-ccf5-41dc-86b1-fa5115caf095","Type":"ContainerStarted","Data":"aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a"}
Sep 11 18:23:40.839873 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.839758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn" event={"ID":"17abdb1f-ccf5-41dc-86b1-fa5115caf095","Type":"ContainerStarted","Data":"4beb71ac4657ab4c6e1cbcc28e7cd5699963678e97eaa9f863128d129add3067"}
Sep 11 18:23:40.839873 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.839780    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:40.853919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.853847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:40.857791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.857736    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" podStartSLOduration=2.481410059 podStartE2EDuration="2.857715822s" totalImagesPullingTime="376.305763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:40.856562495 +0000 UTC m=+789.449763076" watchObservedRunningTime="2026-09-11 18:23:40.857715822 +0000 UTC m=+789.450916380"
Sep 11 18:23:40.875481 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:40.875426    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn" podStartSLOduration=1.875409315 podStartE2EDuration="1.875409315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:40.873338613 +0000 UTC m=+789.466539175" watchObservedRunningTime="2026-09-11 18:23:40.875409315 +0000 UTC m=+789.468609868"
Sep 11 18:23:41.235296 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:41.235176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:41.324672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:41.324626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:41.707996 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:41.707954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:41.755144 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:41.755103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:23:41.820285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:41.816125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.154720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.154688    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.258374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.258343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.353780 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.353743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.648467 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.648430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.748960 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.748851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.761578 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.761550    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:23:42.860514 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.860480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:42.965614 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:42.965569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:43.032467 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:43.032436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:43.140728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:43.140691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:43.325327 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:43.325226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:43.953564 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:43.953528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:44.774434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:44.774397    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:23:44.782473 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:44.782442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:45.704718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:45.704684    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:45.814908 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:45.814872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:46.582028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.581996    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:46.813429 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.813396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:46.851224 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.851141    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:23:46.880697 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.880650    2563 generic.go:356] "Generic (PLEG): container finished" podID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerID="0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480" exitCode=0
Sep 11 18:23:46.881045 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.880903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" event={"ID":"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067","Type":"ContainerDied","Data":"0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"}
Sep 11 18:23:46.881561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.881543    2563 scope.go:117] "RemoveContainer" containerID="0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"
Sep 11 18:23:46.883097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.883056    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:23:46.883194 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.883158    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:23:46.883239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:46.883208    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:23:47.167046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:47.166958    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:47.287447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:47.287410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:47.384211 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:47.384174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:47.572470 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:47.572420    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:47.674556 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:47.674521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:47.890632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:47.890539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" event={"ID":"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067","Type":"ContainerStarted","Data":"0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463"}
Sep 11 18:23:48.481561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:48.481518    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:48.481782 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:48.481759    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 18:23:48.481829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:48.481809    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 18:23:48.722247 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:48.722168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:49.481130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:49.481094    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 18:23:49.481636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:49.481166    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 18:23:49.784620 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:49.784567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:50.481066 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:50.481017    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 18:23:50.481301 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:50.481082    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 18:23:50.626421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:50.626386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:51.481007 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:51.480970    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 18:23:51.481198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:51.481039    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 18:23:51.775648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:51.775610    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:51.847663 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:51.847631    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:23:51.882875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:51.882607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:52.170594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.170507    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:52.348443 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.348379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:52.452681 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.452586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:52.481572 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.481528    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 18:23:52.481749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.481594    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 18:23:52.507285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.502440    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"]
Sep 11 18:23:52.507285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.506217    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.509017 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.508984    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-12eb53-dockercfg-bdb8h\""
Sep 11 18:23:52.509407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.509383    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-12eb53\""
Sep 11 18:23:52.534929 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.534530    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"]
Sep 11 18:23:52.578450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.578412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmb2c\" (UniqueName: \"kubernetes.io/projected/13fbfc94-e5a4-4a79-bd4e-12e757c81568-kube-api-access-vmb2c\") pod \"payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.578638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.578470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13fbfc94-e5a4-4a79-bd4e-12e757c81568-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.679634 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.679589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmb2c\" (UniqueName: \"kubernetes.io/projected/13fbfc94-e5a4-4a79-bd4e-12e757c81568-kube-api-access-vmb2c\") pod \"payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.679843 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.679657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13fbfc94-e5a4-4a79-bd4e-12e757c81568-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.680417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.680389    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13fbfc94-e5a4-4a79-bd4e-12e757c81568-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.688818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.688792    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmb2c\" (UniqueName: \"kubernetes.io/projected/13fbfc94-e5a4-4a79-bd4e-12e757c81568-kube-api-access-vmb2c\") pod \"payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") " pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:52.819331 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.819289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:52.840008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:52.839971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:53.131349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.131321    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"]
Sep 11 18:23:53.133886 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:23:53.133840    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13fbfc94_e5a4_4a79_bd4e_12e757c81568.slice/crio-fd2e00951381c543024f6ba13e65ee087db5922900376eed0066a294bccae6db WatchSource:0}: Error finding container fd2e00951381c543024f6ba13e65ee087db5922900376eed0066a294bccae6db: Status 404 returned error can't find the container with id fd2e00951381c543024f6ba13e65ee087db5922900376eed0066a294bccae6db
Sep 11 18:23:53.481363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.480926    2563 patch_prober.go:28] interesting pod/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 18:23:53.481363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.480988    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 18:23:53.793658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.793623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:53.952043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.951998    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq" event={"ID":"13fbfc94-e5a4-4a79-bd4e-12e757c81568","Type":"ContainerStarted","Data":"eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9"}
Sep 11 18:23:53.952542 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.952055    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq" event={"ID":"13fbfc94-e5a4-4a79-bd4e-12e757c81568","Type":"ContainerStarted","Data":"fd2e00951381c543024f6ba13e65ee087db5922900376eed0066a294bccae6db"}
Sep 11 18:23:53.952542 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.952083    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:23:53.972696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:53.972635    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq" podStartSLOduration=1.972611522 podStartE2EDuration="1.972611522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:23:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:23:53.97162861 +0000 UTC m=+802.564829169" watchObservedRunningTime="2026-09-11 18:23:53.972611522 +0000 UTC m=+802.565812079"
Sep 11 18:23:54.485811 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:54.485774    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:54.486132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:54.486098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:54.486915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:54.486895    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:23:54.502404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:54.502371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:54.860448 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:54.860409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:55.367419 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:55.367387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:56.285063 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:56.285023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:57.063244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:57.063208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:57.721228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:57.721182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:58.565188 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:58.565142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:23:59.308765 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:23:59.308732    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:00.124246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:00.124215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:00.952589 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:00.952544    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:01.696663 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:01.696623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:02.391864 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:02.391827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:04.960395 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:04.960366    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:24:06.090937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.090908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:06.648506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.648478    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"]
Sep 11 18:24:06.648785 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.648752    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" podUID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" containerName="maas-api" containerID="cri-o://24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5" gracePeriod=30
Sep 11 18:24:06.678196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.678173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:06.709529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.709496    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"]
Sep 11 18:24:06.709782 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.709753    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn" podUID="17abdb1f-ccf5-41dc-86b1-fa5115caf095" containerName="payload-processing" containerID="cri-o://aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a" gracePeriod=30
Sep 11 18:24:06.760710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.760685    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:06.845731 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.845692    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" podUID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.49:8443/health\": dial tcp 10.134.0.49:8443: connect: connection refused"
Sep 11 18:24:06.859399 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.859368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:06.961094 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.961067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:06.962422 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.962404    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:24:06.999081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:06.999046    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:24:07.040219 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.040179    2563 generic.go:356] "Generic (PLEG): container finished" podID="17abdb1f-ccf5-41dc-86b1-fa5115caf095" containerID="aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a" exitCode=0
Sep 11 18:24:07.040425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.040303    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"
Sep 11 18:24:07.040425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.040253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn" event={"ID":"17abdb1f-ccf5-41dc-86b1-fa5115caf095","Type":"ContainerDied","Data":"aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a"}
Sep 11 18:24:07.040633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.040580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn" event={"ID":"17abdb1f-ccf5-41dc-86b1-fa5115caf095","Type":"ContainerDied","Data":"4beb71ac4657ab4c6e1cbcc28e7cd5699963678e97eaa9f863128d129add3067"}
Sep 11 18:24:07.040709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.040658    2563 scope.go:117] "RemoveContainer" containerID="aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a"
Sep 11 18:24:07.042251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.042221    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" containerID="24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5" exitCode=0
Sep 11 18:24:07.042434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.042414    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"
Sep 11 18:24:07.042666 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.042646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" event={"ID":"c7f8341e-c86d-41ed-9d71-5aee54d051d2","Type":"ContainerDied","Data":"24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5"}
Sep 11 18:24:07.042749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.042689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g" event={"ID":"c7f8341e-c86d-41ed-9d71-5aee54d051d2","Type":"ContainerDied","Data":"2c9b164d34ac1dad2ed31beb93456ba54b8693ff772f50b44234fb73827fa739"}
Sep 11 18:24:07.053984 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.053961    2563 scope.go:117] "RemoveContainer" containerID="aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a"
Sep 11 18:24:07.054477 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:24:07.054453    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a\": container with ID starting with aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a not found: ID does not exist" containerID="aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a"
Sep 11 18:24:07.054581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.054484    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a"} err="failed to get container status \"aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a\": rpc error: code = NotFound desc = could not find container \"aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a\": container with ID starting with aae5b465404cebb221bbe391dda6f7b09ff9d2a207386851c63be5826ccfed0a not found: ID does not exist"
Sep 11 18:24:07.054581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.054501    2563 scope.go:117] "RemoveContainer" containerID="24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5"
Sep 11 18:24:07.064419 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.064386    2563 scope.go:117] "RemoveContainer" containerID="24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5"
Sep 11 18:24:07.064669 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:24:07.064653    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5\": container with ID starting with 24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5 not found: ID does not exist" containerID="24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5"
Sep 11 18:24:07.064735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.064676    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5"} err="failed to get container status \"24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5\": rpc error: code = NotFound desc = could not find container \"24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5\": container with ID starting with 24470ccf79e6254cde277b8e6ad3162d8798f5aeb84ba7b9a2153c17bcfda3a5 not found: ID does not exist"
Sep 11 18:24:07.136475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136440    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g9fp\" (UniqueName: \"kubernetes.io/projected/17abdb1f-ccf5-41dc-86b1-fa5115caf095-kube-api-access-2g9fp\") pod \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") "
Sep 11 18:24:07.136475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-metrics-certs\") pod \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") "
Sep 11 18:24:07.136982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f8341e-c86d-41ed-9d71-5aee54d051d2-openshift-service-ca\") pod \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") "
Sep 11 18:24:07.136982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17abdb1f-ccf5-41dc-86b1-fa5115caf095-plugins-config-volume\") pod \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\" (UID: \"17abdb1f-ccf5-41dc-86b1-fa5115caf095\") "
Sep 11 18:24:07.136982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136595    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls\") pod \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") "
Sep 11 18:24:07.136982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136623    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjzhl\" (UniqueName: \"kubernetes.io/projected/c7f8341e-c86d-41ed-9d71-5aee54d051d2-kube-api-access-bjzhl\") pod \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\" (UID: \"c7f8341e-c86d-41ed-9d71-5aee54d051d2\") "
Sep 11 18:24:07.136982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136896    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f8341e-c86d-41ed-9d71-5aee54d051d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c7f8341e-c86d-41ed-9d71-5aee54d051d2" (UID: "c7f8341e-c86d-41ed-9d71-5aee54d051d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:24:07.136982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.136931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17abdb1f-ccf5-41dc-86b1-fa5115caf095-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "17abdb1f-ccf5-41dc-86b1-fa5115caf095" (UID: "17abdb1f-ccf5-41dc-86b1-fa5115caf095"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:24:07.137332 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.137136    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7f8341e-c86d-41ed-9d71-5aee54d051d2-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:07.137332 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.137157    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17abdb1f-ccf5-41dc-86b1-fa5115caf095-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:07.139183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.139148    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c7f8341e-c86d-41ed-9d71-5aee54d051d2" (UID: "c7f8341e-c86d-41ed-9d71-5aee54d051d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:24:07.139429 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.139199    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c7f8341e-c86d-41ed-9d71-5aee54d051d2" (UID: "c7f8341e-c86d-41ed-9d71-5aee54d051d2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:24:07.139429 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.139401    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f8341e-c86d-41ed-9d71-5aee54d051d2-kube-api-access-bjzhl" (OuterVolumeSpecName: "kube-api-access-bjzhl") pod "c7f8341e-c86d-41ed-9d71-5aee54d051d2" (UID: "c7f8341e-c86d-41ed-9d71-5aee54d051d2"). InnerVolumeSpecName "kube-api-access-bjzhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:24:07.139554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.139425    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17abdb1f-ccf5-41dc-86b1-fa5115caf095-kube-api-access-2g9fp" (OuterVolumeSpecName: "kube-api-access-2g9fp") pod "17abdb1f-ccf5-41dc-86b1-fa5115caf095" (UID: "17abdb1f-ccf5-41dc-86b1-fa5115caf095"). InnerVolumeSpecName "kube-api-access-2g9fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:24:07.150515 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.150492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:07.238047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.237967    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:07.238047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.237995    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzhl\" (UniqueName: \"kubernetes.io/projected/c7f8341e-c86d-41ed-9d71-5aee54d051d2-kube-api-access-bjzhl\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:07.238047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.238006    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g9fp\" (UniqueName: \"kubernetes.io/projected/17abdb1f-ccf5-41dc-86b1-fa5115caf095-kube-api-access-2g9fp\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:07.238047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.238015    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c7f8341e-c86d-41ed-9d71-5aee54d051d2-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:07.368869 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.368831    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"]
Sep 11 18:24:07.374092 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.374063    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2f7a9131-77ff96c8c-454jn"]
Sep 11 18:24:07.384814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.384786    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"]
Sep 11 18:24:07.389989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.389960    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2f7a9131-78985c4464-vdt2g"]
Sep 11 18:24:07.408169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.408147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:07.510176 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.510088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:07.608448 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.608421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:07.708056 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.708025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:07.914136 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.914107    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17abdb1f-ccf5-41dc-86b1-fa5115caf095" path="/var/lib/kubelet/pods/17abdb1f-ccf5-41dc-86b1-fa5115caf095/volumes"
Sep 11 18:24:07.914560 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.914546    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" path="/var/lib/kubelet/pods/c7f8341e-c86d-41ed-9d71-5aee54d051d2/volumes"
Sep 11 18:24:07.989163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:07.989056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:08.005967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.005941    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"]
Sep 11 18:24:08.006539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.006517    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17abdb1f-ccf5-41dc-86b1-fa5115caf095" containerName="payload-processing"
Sep 11 18:24:08.006539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.006539    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17abdb1f-ccf5-41dc-86b1-fa5115caf095" containerName="payload-processing"
Sep 11 18:24:08.006728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.006561    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" containerName="maas-api"
Sep 11 18:24:08.006728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.006569    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" containerName="maas-api"
Sep 11 18:24:08.006728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.006659    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17abdb1f-ccf5-41dc-86b1-fa5115caf095" containerName="payload-processing"
Sep 11 18:24:08.006728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.006671    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f8341e-c86d-41ed-9d71-5aee54d051d2" containerName="maas-api"
Sep 11 18:24:08.010693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.010675    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.013066 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.013048    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f8c01007-gw-openshift-default-dockercfg-2fcmk\""
Sep 11 18:24:08.023593 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.023570    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"]
Sep 11 18:24:08.079090 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.079050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:08.146983 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.146949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-socket\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.146988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-podinfo\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-certs\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-data\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-envoy\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b94cee2e-c31f-454a-879f-d45e4aa61aea-istiod-ca-cert\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-credential-socket\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-token\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.147427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.147295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqb9\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-kube-api-access-zqqb9\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.216426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.216350    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:08.248324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248296    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-envoy\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b94cee2e-c31f-454a-879f-d45e4aa61aea-istiod-ca-cert\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-credential-socket\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-token\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqb9\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-kube-api-access-zqqb9\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248667 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-socket\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248719 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-podinfo\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248784 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-certs\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-credential-socket\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-data\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.248851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.248830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-socket\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.249084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.249062    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b94cee2e-c31f-454a-879f-d45e4aa61aea-istiod-ca-cert\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.249164 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.249145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-data\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.249217 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.249196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-certs\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.250922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.250896    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-envoy\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.251201 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.251181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-podinfo\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.258390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.258365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqb9\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-kube-api-access-zqqb9\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.259355 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.259330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-token\") pod \"e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") " pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.317125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.317096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:08.323229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.323204    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:08.488605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.488519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:08.600126 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.600093    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"]
Sep 11 18:24:08.601537 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:24:08.601512    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94cee2e_c31f_454a_879f_d45e4aa61aea.slice/crio-c2f8a66ea804b6aac5b638e1c147cf58df798e30685e6832c05a1e7e4d9136e1 WatchSource:0}: Error finding container c2f8a66ea804b6aac5b638e1c147cf58df798e30685e6832c05a1e7e4d9136e1: Status 404 returned error can't find the container with id c2f8a66ea804b6aac5b638e1c147cf58df798e30685e6832c05a1e7e4d9136e1
Sep 11 18:24:08.604081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.604052    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:24:08.604140 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.604114    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:24:08.604177 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:08.604142    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:24:09.060485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:09.060449    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" event={"ID":"b94cee2e-c31f-454a-879f-d45e4aa61aea","Type":"ContainerStarted","Data":"d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"}
Sep 11 18:24:09.060659 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:09.060510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" event={"ID":"b94cee2e-c31f-454a-879f-d45e4aa61aea","Type":"ContainerStarted","Data":"c2f8a66ea804b6aac5b638e1c147cf58df798e30685e6832c05a1e7e4d9136e1"}
Sep 11 18:24:09.080519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:09.080476    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podStartSLOduration=2.08046119 podStartE2EDuration="2.08046119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:24:09.079313703 +0000 UTC m=+817.672514262" watchObservedRunningTime="2026-09-11 18:24:09.08046119 +0000 UTC m=+817.673661747"
Sep 11 18:24:09.216314 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:09.216284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:09.323883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:09.323808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:09.328718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:09.328694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:10.001946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:10.001910    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:10.067644 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:10.067604    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:10.068885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:10.068853    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:10.791214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:10.791180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:10.891449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:10.891418    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:11.693131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:11.693098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:12.076685 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:12.076648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:12.362110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:12.362032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:13.162170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:13.162140    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:14.079458 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:14.079427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:14.237654 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:14.237621    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:14.381113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:14.381042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:14.718187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:14.718051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:14.810330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:14.810300    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:15.179369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:15.179336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:15.888694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:15.888656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:16.101426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:16.101396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:16.203795 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:16.203728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:16.301033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:16.301007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:16.459006 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:16.458938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:16.803228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:16.803198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:23.384189 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:23.384154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:23.448884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:23.448852    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:29.209550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.209511    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"]
Sep 11 18:24:29.219155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.218239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.220881 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.220835    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"]
Sep 11 18:24:29.221450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.221408    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-d73031\"/\"kube-root-ca.crt\""
Sep 11 18:24:29.222273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.222066    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d73031\"/\"default-dockercfg-n6mnk\""
Sep 11 18:24:29.222273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.222101    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-d73031\"/\"openshift-service-ca.crt\""
Sep 11 18:24:29.222273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.222128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d73031\"/\"simulator-w0-d73031-kserve-self-signed-certs\""
Sep 11 18:24:29.227580 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.227488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:29.355370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.355319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-tmp-dir\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.355562 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.355452    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-home\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.355562 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.355492    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgzl\" (UniqueName: \"kubernetes.io/projected/a1c08e9c-9742-45e1-b067-911205bc3a88-kube-api-access-mtgzl\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.355562 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.355524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-dshm\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.355562 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.355545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c08e9c-9742-45e1-b067-911205bc3a88-tls-certs\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.355747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.355570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-model-cache\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.456752 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.456715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-model-cache\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.456957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.456824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-tmp-dir\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.456957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.456888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-home\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.456957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.456933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgzl\" (UniqueName: \"kubernetes.io/projected/a1c08e9c-9742-45e1-b067-911205bc3a88-kube-api-access-mtgzl\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.457128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.456965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-dshm\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.457128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.456987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c08e9c-9742-45e1-b067-911205bc3a88-tls-certs\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.457234 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.457126    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-model-cache\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.457234 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.457179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-home\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.457365 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.457237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-tmp-dir\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.459306 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.459248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-dshm\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.459657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.459605    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c08e9c-9742-45e1-b067-911205bc3a88-tls-certs\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.466051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.466025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgzl\" (UniqueName: \"kubernetes.io/projected/a1c08e9c-9742-45e1-b067-911205bc3a88-kube-api-access-mtgzl\") pod \"simulator-w0-d73031-kserve-5d767f49cf-z72xv\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") " pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.532679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.532653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:29.808097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:29.808070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"]
Sep 11 18:24:29.810557 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:24:29.810522    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1c08e9c_9742_45e1_b067_911205bc3a88.slice/crio-ad02a80eacd11326936a7e4f7381a6c86ec114172ad97a0f1e916fd6dd02e54e WatchSource:0}: Error finding container ad02a80eacd11326936a7e4f7381a6c86ec114172ad97a0f1e916fd6dd02e54e: Status 404 returned error can't find the container with id ad02a80eacd11326936a7e4f7381a6c86ec114172ad97a0f1e916fd6dd02e54e
Sep 11 18:24:30.175416 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:30.175343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv" event={"ID":"a1c08e9c-9742-45e1-b067-911205bc3a88","Type":"ContainerStarted","Data":"4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9"}
Sep 11 18:24:30.175416 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:30.175383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv" event={"ID":"a1c08e9c-9742-45e1-b067-911205bc3a88","Type":"ContainerStarted","Data":"ad02a80eacd11326936a7e4f7381a6c86ec114172ad97a0f1e916fd6dd02e54e"}
Sep 11 18:24:30.194513 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:30.194467    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv" podStartSLOduration=1.194452254 podStartE2EDuration="1.194452254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:24:30.19312643 +0000 UTC m=+838.786326986" watchObservedRunningTime="2026-09-11 18:24:30.194452254 +0000 UTC m=+838.787652811"
Sep 11 18:24:31.182775 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:31.182750    2563 generic.go:356] "Generic (PLEG): container finished" podID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerID="d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43" exitCode=0
Sep 11 18:24:31.183089 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:31.182819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" event={"ID":"b94cee2e-c31f-454a-879f-d45e4aa61aea","Type":"ContainerDied","Data":"d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"}
Sep 11 18:24:31.183388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:31.183372    2563 scope.go:117] "RemoveContainer" containerID="d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"
Sep 11 18:24:31.184401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:31.184374    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:24:31.184457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:31.184439    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:24:31.184494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:31.184468    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:24:32.190327 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:32.190244    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" event={"ID":"b94cee2e-c31f-454a-879f-d45e4aa61aea","Type":"ContainerStarted","Data":"55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c"}
Sep 11 18:24:32.323447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:32.323419    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:32.323621 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:32.323603    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 18:24:32.323669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:32.323644    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 18:24:33.324427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:33.324385    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 18:24:33.324803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:33.324447    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 18:24:34.323993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:34.323955    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 18:24:34.324695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:34.324659    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 18:24:35.324276 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:35.324216    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 18:24:35.324421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:35.324313    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 18:24:36.324000 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:36.323961    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 18:24:36.324407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:36.324023    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 18:24:37.164590 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:37.164551    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:37.323971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:37.323937    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 18:24:37.324252 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:37.324068    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 18:24:38.114822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:38.114770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:38.356255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:38.356225    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:38.356756 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:38.356623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:38.357781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:38.357760    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:38.906036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:38.905994    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:39.533816 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:39.533783    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:39.534292 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:39.533826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:39.546086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:39.546060    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:39.701268 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:39.701237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:40.255169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:40.255139    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:24:40.528214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:40.528183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:41.353024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:41.352990    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:42.176588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:42.176550    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:42.942152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:42.942115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:43.639955 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:43.639923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:44.382868 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:44.382827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:45.077354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:45.077321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:45.888879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:45.888818    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:51.928963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:51.928924    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:52.509734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.509697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:52.577968 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.577933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:52.639338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.639305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:52.709798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.709774    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:52.778231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.778200    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:52.848613 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.848581    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"]
Sep 11 18:24:52.848865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.848840    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" containerID="cri-o://55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c" gracePeriod=30
Sep 11 18:24:52.864032 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:52.864004    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:53.107392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.107315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:53.280622 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.280595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:53.358589 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.358507    2563 patch_prober.go:28] interesting pod/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:24:53.358589 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.358565    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:24:53.587474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.587443    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:53.667881 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.667795    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:53.988436 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:53.988353    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:54.809921 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:54.809890    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:55.088222 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:55.088121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:55.598420 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:55.598387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:56.132032 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:56.131998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:56.232459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:56.232421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:56.481845 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:56.481745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:57.163298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:57.163239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:57.900701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:57.900658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:58.001409 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.001378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:58.107100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.107077    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:58.134642 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134579    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b94cee2e-c31f-454a-879f-d45e4aa61aea-istiod-ca-cert\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.134642 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134615    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-podinfo\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.134832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134651    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-certs\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.134832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134681    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-socket\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.134832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134752    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-token\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.134832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134800    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqqb9\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-kube-api-access-zqqb9\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.135046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134872    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-credential-socket\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.135046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-envoy\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.135046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.134938    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-data\") pod \"b94cee2e-c31f-454a-879f-d45e4aa61aea\" (UID: \"b94cee2e-c31f-454a-879f-d45e4aa61aea\") "
Sep 11 18:24:58.135046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.135016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b94cee2e-c31f-454a-879f-d45e4aa61aea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:24:58.135566 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.135306    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:24:58.135566 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.135343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:24:58.135566 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.135356    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b94cee2e-c31f-454a-879f-d45e4aa61aea-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.135799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.135574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-data" (OuterVolumeSpecName: "istio-data") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:24:58.135799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.135714    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:24:58.137178 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.137149    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:24:58.138138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.138101    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-kube-api-access-zqqb9" (OuterVolumeSpecName: "kube-api-access-zqqb9") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "kube-api-access-zqqb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:24:58.138138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.138107    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-token" (OuterVolumeSpecName: "istio-token") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:24:58.138331 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.138297    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b94cee2e-c31f-454a-879f-d45e4aa61aea" (UID: "b94cee2e-c31f-454a-879f-d45e4aa61aea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:24:58.236771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236744    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.236771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236771    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqqb9\" (UniqueName: \"kubernetes.io/projected/b94cee2e-c31f-454a-879f-d45e4aa61aea-kube-api-access-zqqb9\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.237133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236786    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.237133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236799    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.237133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236811    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.237133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236824    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b94cee2e-c31f-454a-879f-d45e4aa61aea-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.237133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236836    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.237133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.236849    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b94cee2e-c31f-454a-879f-d45e4aa61aea-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:24:58.306941 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.306915    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:58.338438 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.338409    2563 generic.go:356] "Generic (PLEG): container finished" podID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerID="55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c" exitCode=0
Sep 11 18:24:58.338554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.338459    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"
Sep 11 18:24:58.338554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.338494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" event={"ID":"b94cee2e-c31f-454a-879f-d45e4aa61aea","Type":"ContainerDied","Data":"55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c"}
Sep 11 18:24:58.338554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.338546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52" event={"ID":"b94cee2e-c31f-454a-879f-d45e4aa61aea","Type":"ContainerDied","Data":"c2f8a66ea804b6aac5b638e1c147cf58df798e30685e6832c05a1e7e4d9136e1"}
Sep 11 18:24:58.338702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.338567    2563 scope.go:117] "RemoveContainer" containerID="55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c"
Sep 11 18:24:58.350167 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.349463    2563 scope.go:117] "RemoveContainer" containerID="d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"
Sep 11 18:24:58.359724 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.359609    2563 scope.go:117] "RemoveContainer" containerID="55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c"
Sep 11 18:24:58.360348 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:24:58.360039    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c\": container with ID starting with 55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c not found: ID does not exist" containerID="55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c"
Sep 11 18:24:58.360348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.360077    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c"} err="failed to get container status \"55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c\": rpc error: code = NotFound desc = could not find container \"55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c\": container with ID starting with 55abbe13d71c5263aa76be3f69309faccc3fd1b332118a319bc2b38c2afa891c not found: ID does not exist"
Sep 11 18:24:58.360348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.360100    2563 scope.go:117] "RemoveContainer" containerID="d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"
Sep 11 18:24:58.360552 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:24:58.360432    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43\": container with ID starting with d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43 not found: ID does not exist" containerID="d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"
Sep 11 18:24:58.360552 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.360461    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43"} err="failed to get container status \"d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43\": rpc error: code = NotFound desc = could not find container \"d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43\": container with ID starting with d4f8acd69ab1ea240ad4da0432e8b84f2e6440f0b3efbbc0886ada53c5daae43 not found: ID does not exist"
Sep 11 18:24:58.362383 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.362364    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"]
Sep 11 18:24:58.368516 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.368489    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f8c01007-gw-openshift-default-6db5545dcb-49d52"]
Sep 11 18:24:58.462552 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.462487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:58.559897 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.559870    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:58.574512 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.574486    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"]
Sep 11 18:24:58.574909 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.574898    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy"
Sep 11 18:24:58.574956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.574911    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy"
Sep 11 18:24:58.574956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.574921    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy"
Sep 11 18:24:58.574956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.574927    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy"
Sep 11 18:24:58.575050 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.575003    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy"
Sep 11 18:24:58.579672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.579657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.582144 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.582123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ed21d135-dockercfg-km74q\""
Sep 11 18:24:58.582250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.582180    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ed21d135\""
Sep 11 18:24:58.587347 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.587274    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"]
Sep 11 18:24:58.639746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.639718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb47j\" (UniqueName: \"kubernetes.io/projected/3f3007e0-78c5-43b6-a056-c5954f10d3d6-kube-api-access-tb47j\") pod \"payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.639864 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.639810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f3007e0-78c5-43b6-a056-c5954f10d3d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.740984 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.740875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb47j\" (UniqueName: \"kubernetes.io/projected/3f3007e0-78c5-43b6-a056-c5954f10d3d6-kube-api-access-tb47j\") pod \"payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.740984 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.740958    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f3007e0-78c5-43b6-a056-c5954f10d3d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.741617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.741593    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f3007e0-78c5-43b6-a056-c5954f10d3d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.749047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.749018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb47j\" (UniqueName: \"kubernetes.io/projected/3f3007e0-78c5-43b6-a056-c5954f10d3d6-kube-api-access-tb47j\") pod \"payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:58.862234 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.862208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:58.891655 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:58.891629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:59.164447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.164419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"]
Sep 11 18:24:59.165477 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:24:59.165454    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3007e0_78c5_43b6_a056_c5954f10d3d6.slice/crio-17a28122fc0d9643613ceb22f7bbfadbeb6a6f3458392ec44f571aaf7f4872d9 WatchSource:0}: Error finding container 17a28122fc0d9643613ceb22f7bbfadbeb6a6f3458392ec44f571aaf7f4872d9: Status 404 returned error can't find the container with id 17a28122fc0d9643613ceb22f7bbfadbeb6a6f3458392ec44f571aaf7f4872d9
Sep 11 18:24:59.346820 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.346785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx" event={"ID":"3f3007e0-78c5-43b6-a056-c5954f10d3d6","Type":"ContainerStarted","Data":"de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883"}
Sep 11 18:24:59.347190 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.346831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx" event={"ID":"3f3007e0-78c5-43b6-a056-c5954f10d3d6","Type":"ContainerStarted","Data":"17a28122fc0d9643613ceb22f7bbfadbeb6a6f3458392ec44f571aaf7f4872d9"}
Sep 11 18:24:59.347190 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.346931    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:24:59.367357 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.367312    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx" podStartSLOduration=1.367299376 podStartE2EDuration="1.367299376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:24:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:24:59.365185905 +0000 UTC m=+867.958386462" watchObservedRunningTime="2026-09-11 18:24:59.367299376 +0000 UTC m=+867.960499933"
Sep 11 18:24:59.582585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.582556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:24:59.921875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:24:59.921787    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" path="/var/lib/kubelet/pods/b94cee2e-c31f-454a-879f-d45e4aa61aea/volumes"
Sep 11 18:25:00.393203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:00.393173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:01.167784 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:01.167751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:10.354401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:10.354368    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:25:19.951971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:19.951923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:20.688493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:20.688463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:21.409054 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.409024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:21.814384 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.814354    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"]
Sep 11 18:25:21.814955 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.814936    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b94cee2e-c31f-454a-879f-d45e4aa61aea" containerName="istio-proxy"
Sep 11 18:25:21.819673 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.819657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:21.821859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.821820    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t9kdx\""
Sep 11 18:25:21.826337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.826312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"]
Sep 11 18:25:21.948467 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:21.948433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e8cc91-13a5-404d-8844-6677f95cfa73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed21d135-vfp85\" (UID: \"f8e8cc91-13a5-404d-8844-6677f95cfa73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:22.049513 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:22.049479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e8cc91-13a5-404d-8844-6677f95cfa73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed21d135-vfp85\" (UID: \"f8e8cc91-13a5-404d-8844-6677f95cfa73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:22.050172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:22.050146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e8cc91-13a5-404d-8844-6677f95cfa73-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-ed21d135-vfp85\" (UID: \"f8e8cc91-13a5-404d-8844-6677f95cfa73\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:22.154312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:22.154227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:22.350462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:22.350414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:22.425106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:22.425078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"]
Sep 11 18:25:22.427524 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:25:22.427501    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e8cc91_13a5_404d_8844_6677f95cfa73.slice/crio-0cf865861163e4f13ebc9bd8c0244ecce7861d696127db7c93b725e490bdd664 WatchSource:0}: Error finding container 0cf865861163e4f13ebc9bd8c0244ecce7861d696127db7c93b725e490bdd664: Status 404 returned error can't find the container with id 0cf865861163e4f13ebc9bd8c0244ecce7861d696127db7c93b725e490bdd664
Sep 11 18:25:22.471039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:22.471016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85" event={"ID":"f8e8cc91-13a5-404d-8844-6677f95cfa73","Type":"ContainerStarted","Data":"0cf865861163e4f13ebc9bd8c0244ecce7861d696127db7c93b725e490bdd664"}
Sep 11 18:25:23.161749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:23.161682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:23.874156 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:23.874115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:24.612641 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:24.612607    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:25.488957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:25.488923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:25.489966 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:25.489935    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85" event={"ID":"f8e8cc91-13a5-404d-8844-6677f95cfa73","Type":"ContainerStarted","Data":"0ffcd0b1578e62c09b3a986b6942c2995c0f1c1ec7a34f833026bde733de38aa"}
Sep 11 18:25:25.505912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:25.505867    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85" podStartSLOduration=1.9189111890000001 podStartE2EDuration="4.505853112s" totalImagesPullingTime="2.586941923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:25:25.50543061 +0000 UTC m=+894.098631167" watchObservedRunningTime="2026-09-11 18:25:25.505853112 +0000 UTC m=+894.099053668"
Sep 11 18:25:26.162910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:26.162881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:26.496955 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:26.496881    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8e8cc91-13a5-404d-8844-6677f95cfa73" containerID="0ffcd0b1578e62c09b3a986b6942c2995c0f1c1ec7a34f833026bde733de38aa" exitCode=0
Sep 11 18:25:26.496955 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:26.496946    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85" event={"ID":"f8e8cc91-13a5-404d-8844-6677f95cfa73","Type":"ContainerDied","Data":"0ffcd0b1578e62c09b3a986b6942c2995c0f1c1ec7a34f833026bde733de38aa"}
Sep 11 18:25:26.992322 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:26.992289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:27.634072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:27.634046    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:27.798960 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:27.798937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e8cc91-13a5-404d-8844-6677f95cfa73-maas-api-service-ca\") pod \"f8e8cc91-13a5-404d-8844-6677f95cfa73\" (UID: \"f8e8cc91-13a5-404d-8844-6677f95cfa73\") "
Sep 11 18:25:27.799308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:27.799248    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8e8cc91-13a5-404d-8844-6677f95cfa73-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f8e8cc91-13a5-404d-8844-6677f95cfa73" (UID: "f8e8cc91-13a5-404d-8844-6677f95cfa73"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:25:27.900196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:27.900172    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8e8cc91-13a5-404d-8844-6677f95cfa73-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:25:28.297210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.297184    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"]
Sep 11 18:25:28.297749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.297729    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8e8cc91-13a5-404d-8844-6677f95cfa73" containerName="revoke-keys"
Sep 11 18:25:28.297749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.297750    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8e8cc91-13a5-404d-8844-6677f95cfa73" containerName="revoke-keys"
Sep 11 18:25:28.297949 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.297859    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8e8cc91-13a5-404d-8844-6677f95cfa73" containerName="revoke-keys"
Sep 11 18:25:28.300942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.300921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.304011 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.303973    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-89bb6c\"/\"default-dockercfg-jhwz9\""
Sep 11 18:25:28.304011 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.303989    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-89bb6c\"/\"openshift-service-ca.crt\""
Sep 11 18:25:28.304187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.304031    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-89bb6c\"/\"kube-root-ca.crt\""
Sep 11 18:25:28.304187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.303979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-89bb6c\"/\"autores-model-89bb6c-kserve-self-signed-certs\""
Sep 11 18:25:28.311835 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.311812    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"]
Sep 11 18:25:28.319646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.319591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:28.404279 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.404239    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tmp-dir\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.404390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.404296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-model-cache\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.404390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.404375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-home\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.404469 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.404407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpkgp\" (UniqueName: \"kubernetes.io/projected/0a0d4e95-df58-45d1-a824-a0effb83b4f1-kube-api-access-rpkgp\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.404507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.404485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-dshm\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.404542 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.404521    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tls-certs\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.505636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.505606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tmp-dir\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.505798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.505643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-model-cache\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.505798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.505681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-home\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.505798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.505704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpkgp\" (UniqueName: \"kubernetes.io/projected/0a0d4e95-df58-45d1-a824-a0effb83b4f1-kube-api-access-rpkgp\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.505798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.505784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-dshm\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.506004 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.505831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tls-certs\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.506124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.506100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-model-cache\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.506192 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.506156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-home\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.506250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.506188    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tmp-dir\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.508181 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.508157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-dshm\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.508403 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.508384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tls-certs\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.509488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.509470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85" event={"ID":"f8e8cc91-13a5-404d-8844-6677f95cfa73","Type":"ContainerDied","Data":"0cf865861163e4f13ebc9bd8c0244ecce7861d696127db7c93b725e490bdd664"}
Sep 11 18:25:28.509569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.509509    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cf865861163e4f13ebc9bd8c0244ecce7861d696127db7c93b725e490bdd664"
Sep 11 18:25:28.509569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.509477    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"
Sep 11 18:25:28.513991 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.513968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpkgp\" (UniqueName: \"kubernetes.io/projected/0a0d4e95-df58-45d1-a824-a0effb83b4f1-kube-api-access-rpkgp\") pod \"autores-model-89bb6c-kserve-8d98c5d67-lmxxw\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") " pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.610801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.610735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:28.883313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:28.883288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"]
Sep 11 18:25:28.884808 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:25:28.884784    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a0d4e95_df58_45d1_a824_a0effb83b4f1.slice/crio-99a06e9de64d812b66182034413f43b82afbf77f9316e261fd1f2773b47394c9 WatchSource:0}: Error finding container 99a06e9de64d812b66182034413f43b82afbf77f9316e261fd1f2773b47394c9: Status 404 returned error can't find the container with id 99a06e9de64d812b66182034413f43b82afbf77f9316e261fd1f2773b47394c9
Sep 11 18:25:29.516492 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:29.516456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw" event={"ID":"0a0d4e95-df58-45d1-a824-a0effb83b4f1","Type":"ContainerStarted","Data":"2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9"}
Sep 11 18:25:29.516670 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:29.516505    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw" event={"ID":"0a0d4e95-df58-45d1-a824-a0effb83b4f1","Type":"ContainerStarted","Data":"99a06e9de64d812b66182034413f43b82afbf77f9316e261fd1f2773b47394c9"}
Sep 11 18:25:29.533627 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:29.533580    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw" podStartSLOduration=1.533565615 podStartE2EDuration="1.533565615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:25:29.532493179 +0000 UTC m=+898.125693749" watchObservedRunningTime="2026-09-11 18:25:29.533565615 +0000 UTC m=+898.126766171"
Sep 11 18:25:31.893246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:31.893221    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:25:31.893676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:31.893329    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:25:31.901764 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:31.901735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:37.524276 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:37.524233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:38.168725 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.168696    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:38.204502 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.204474    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"]
Sep 11 18:25:38.204743 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.204713    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx" podUID="3f3007e0-78c5-43b6-a056-c5954f10d3d6" containerName="payload-pre-processing" containerID="cri-o://de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883" gracePeriod=30
Sep 11 18:25:38.269347 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.269319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:38.357070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.357039    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"]
Sep 11 18:25:38.358349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.358323    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:38.359637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.359611    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-ed21d135-vfp85"]
Sep 11 18:25:38.425490 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.425409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:38.456875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.456847    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:25:38.497420 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.497396    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb47j\" (UniqueName: \"kubernetes.io/projected/3f3007e0-78c5-43b6-a056-c5954f10d3d6-kube-api-access-tb47j\") pod \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") "
Sep 11 18:25:38.497570 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.497447    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f3007e0-78c5-43b6-a056-c5954f10d3d6-plugins-config-volume\") pod \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\" (UID: \"3f3007e0-78c5-43b6-a056-c5954f10d3d6\") "
Sep 11 18:25:38.497856 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.497835    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f3007e0-78c5-43b6-a056-c5954f10d3d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f3007e0-78c5-43b6-a056-c5954f10d3d6" (UID: "3f3007e0-78c5-43b6-a056-c5954f10d3d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:25:38.499528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.499501    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f3007e0-78c5-43b6-a056-c5954f10d3d6-kube-api-access-tb47j" (OuterVolumeSpecName: "kube-api-access-tb47j") pod "3f3007e0-78c5-43b6-a056-c5954f10d3d6" (UID: "3f3007e0-78c5-43b6-a056-c5954f10d3d6"). InnerVolumeSpecName "kube-api-access-tb47j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:25:38.566962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.566928    2563 generic.go:356] "Generic (PLEG): container finished" podID="3f3007e0-78c5-43b6-a056-c5954f10d3d6" containerID="de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883" exitCode=0
Sep 11 18:25:38.567348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.566984    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"
Sep 11 18:25:38.567348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.567008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx" event={"ID":"3f3007e0-78c5-43b6-a056-c5954f10d3d6","Type":"ContainerDied","Data":"de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883"}
Sep 11 18:25:38.567348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.567071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx" event={"ID":"3f3007e0-78c5-43b6-a056-c5954f10d3d6","Type":"ContainerDied","Data":"17a28122fc0d9643613ceb22f7bbfadbeb6a6f3458392ec44f571aaf7f4872d9"}
Sep 11 18:25:38.567348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.567093    2563 scope.go:117] "RemoveContainer" containerID="de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883"
Sep 11 18:25:38.576615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.576599    2563 scope.go:117] "RemoveContainer" containerID="de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883"
Sep 11 18:25:38.576875 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:25:38.576858    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883\": container with ID starting with de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883 not found: ID does not exist" containerID="de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883"
Sep 11 18:25:38.576929 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.576882    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883"} err="failed to get container status \"de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883\": rpc error: code = NotFound desc = could not find container \"de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883\": container with ID starting with de612b126262307f326993089efaa3909b5d3e4163b79299150b688e792da883 not found: ID does not exist"
Sep 11 18:25:38.589867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.589847    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"]
Sep 11 18:25:38.594883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.594865    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ed21d135-746d8ccd86-86stx"]
Sep 11 18:25:38.599096 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.599075    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb47j\" (UniqueName: \"kubernetes.io/projected/3f3007e0-78c5-43b6-a056-c5954f10d3d6-kube-api-access-tb47j\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:25:38.599179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.599100    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f3007e0-78c5-43b6-a056-c5954f10d3d6-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:25:38.611703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.611677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:38.611815 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.611724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:38.628397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.628378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:38.816642 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:38.816610    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:39.478625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:39.478592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:39.585683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:39.585655    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:25:39.605070 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:39.605043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:39.892619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:39.892590    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:39.913603 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:39.913574    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f3007e0-78c5-43b6-a056-c5954f10d3d6" path="/var/lib/kubelet/pods/3f3007e0-78c5-43b6-a056-c5954f10d3d6/volumes"
Sep 11 18:25:39.913965 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:39.913951    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8e8cc91-13a5-404d-8844-6677f95cfa73" path="/var/lib/kubelet/pods/f8e8cc91-13a5-404d-8844-6677f95cfa73/volumes"
Sep 11 18:25:40.240327 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.240210    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"]
Sep 11 18:25:40.240790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.240768    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f3007e0-78c5-43b6-a056-c5954f10d3d6" containerName="payload-pre-processing"
Sep 11 18:25:40.240790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.240790    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f3007e0-78c5-43b6-a056-c5954f10d3d6" containerName="payload-pre-processing"
Sep 11 18:25:40.240946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.240891    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f3007e0-78c5-43b6-a056-c5954f10d3d6" containerName="payload-pre-processing"
Sep 11 18:25:40.245535 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.245513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.247989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.247970    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-23f49f11-openshift-default-dockercfg-zzlxc\""
Sep 11 18:25:40.256999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.256972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:40.257452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.257250    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"]
Sep 11 18:25:40.314082 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq4pr\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-kube-api-access-fq4pr\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-socket\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdac51c2-3985-4704-bf7e-9964b46df665-istiod-ca-cert\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-envoy\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-data\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-certs\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-istio-token\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314415    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdac51c2-3985-4704-bf7e-9964b46df665-istio-podinfo\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.314539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.314445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-credential-socket\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq4pr\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-kube-api-access-fq4pr\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-socket\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdac51c2-3985-4704-bf7e-9964b46df665-istiod-ca-cert\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-envoy\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-data\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-certs\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415455    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-istio-token\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdac51c2-3985-4704-bf7e-9964b46df665-istio-podinfo\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-credential-socket\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-socket\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.415964 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.415928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-data\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.416127 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.416104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdac51c2-3985-4704-bf7e-9964b46df665-istiod-ca-cert\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.416206 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.416192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-credential-socket\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.416298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.416192    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-certs\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.417677 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.417657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-envoy\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.417946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.417928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdac51c2-3985-4704-bf7e-9964b46df665-istio-podinfo\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.423330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.423306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-istio-token\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.423413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.423362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq4pr\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-kube-api-access-fq4pr\") pod \"e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") " pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.477599 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.477562    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:40.559706 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.559672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:40.579855 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.579811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:40.843821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.843796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"]
Sep 11 18:25:40.845368 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:25:40.845341    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdac51c2_3985_4704_bf7e_9964b46df665.slice/crio-3be03f00b8d94014d1b2d7df9e96e835ed0487561747881b6e40e1433e8effd4 WatchSource:0}: Error finding container 3be03f00b8d94014d1b2d7df9e96e835ed0487561747881b6e40e1433e8effd4: Status 404 returned error can't find the container with id 3be03f00b8d94014d1b2d7df9e96e835ed0487561747881b6e40e1433e8effd4
Sep 11 18:25:40.847720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.847684    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:25:40.847831 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.847748    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:25:40.847831 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:40.847776    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:25:41.417393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:41.417358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:41.589986 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:41.589947    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" event={"ID":"bdac51c2-3985-4704-bf7e-9964b46df665","Type":"ContainerStarted","Data":"934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"}
Sep 11 18:25:41.590172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:41.590002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" event={"ID":"bdac51c2-3985-4704-bf7e-9964b46df665","Type":"ContainerStarted","Data":"3be03f00b8d94014d1b2d7df9e96e835ed0487561747881b6e40e1433e8effd4"}
Sep 11 18:25:41.615316 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:41.615250    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podStartSLOduration=1.615231991 podStartE2EDuration="1.615231991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:25:41.613123994 +0000 UTC m=+910.206324551" watchObservedRunningTime="2026-09-11 18:25:41.615231991 +0000 UTC m=+910.208432548"
Sep 11 18:25:42.278344 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:42.278313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:42.560059 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:42.559974    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:42.565001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:42.564977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:42.599282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:42.599248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:42.600152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:42.600129    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:25:42.910741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:42.910656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:43.019730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:43.019697    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:43.109432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:43.109401    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:44.022169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:44.022136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:44.771389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:44.771354    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:45.593461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:45.593431    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:46.272748 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:46.272714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:47.094423 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:47.094397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:47.155333 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:47.155301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:47.259913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:47.259876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:47.560797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:47.560765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:47.651636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:47.651606    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:25:48.884423 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:25:48.884393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:04.379528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:04.379495    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:05.135531 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:05.135496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:05.993472 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:05.993440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:06.687900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:06.687869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:06.906586 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:06.906554    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"]
Sep 11 18:26:07.749874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.749789    2563 generic.go:356] "Generic (PLEG): container finished" podID="bdac51c2-3985-4704-bf7e-9964b46df665" containerID="934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604" exitCode=0
Sep 11 18:26:07.749874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.749858    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" event={"ID":"bdac51c2-3985-4704-bf7e-9964b46df665","Type":"ContainerDied","Data":"934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"}
Sep 11 18:26:07.750339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.750324    2563 scope.go:117] "RemoveContainer" containerID="934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"
Sep 11 18:26:07.751653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.751617    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:07.751728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.751708    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:07.751774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.751759    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:07.848089 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:07.848058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:08.311771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:08.311653    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:08.761212 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:08.761131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" event={"ID":"bdac51c2-3985-4704-bf7e-9964b46df665","Type":"ContainerStarted","Data":"dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b"}
Sep 11 18:26:09.110548 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:09.110513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:09.560246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:09.560217    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:26:09.561153 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:09.561123    2563 patch_prober.go:28] interesting pod/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 18:26:09.561299 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:09.561175    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 18:26:09.912973 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:09.912888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:10.560322 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:10.560255    2563 patch_prober.go:28] interesting pod/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 18:26:10.560497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:10.560346    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 18:26:10.629594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:10.629563    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:11.517959 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:11.517891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:11.560509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:11.560479    2563 patch_prober.go:28] interesting pod/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 18:26:11.560654 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:11.560529    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 18:26:12.274110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:12.274081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:12.560270 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:12.560233    2563 patch_prober.go:28] interesting pod/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 18:26:12.560710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:12.560314    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 18:26:12.599514 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:12.599488    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:26:13.128805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:13.128772    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:13.560685 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:13.560648    2563 patch_prober.go:28] interesting pod/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 18:26:13.561055 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:13.560707    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 18:26:14.275851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:14.275822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:14.564543 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:14.564462    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:26:14.565679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:14.565656    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:26:14.922807 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:14.922730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:14.997207 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:14.997181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.085277 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.085230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.167795 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.167758    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.233019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.232939    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.312743 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.312709    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.376481 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.376448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.465163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.465131    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"]
Sep 11 18:26:15.487516 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.487436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.545530 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.545504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.620478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.620436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:15.801612 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:15.801571    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy" containerID="cri-o://dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b" gracePeriod=30
Sep 11 18:26:16.125765 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:16.125685    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:16.221610 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:16.221585    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:16.305889 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:16.305861    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:16.556013 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:16.555972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:17.396657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:17.396602    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:17.600274 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:17.600236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:18.245256 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:18.245216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:18.904370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:18.904338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:19.689425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:19.689389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:20.579609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:20.579582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:21.055907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.055883    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:26:21.139710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139683    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdac51c2-3985-4704-bf7e-9964b46df665-istiod-ca-cert\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.139710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-istio-token\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.139892 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139768    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq4pr\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-kube-api-access-fq4pr\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.139892 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-credential-socket\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.139892 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139856    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-envoy\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.140053 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-data\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.140053 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139929    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdac51c2-3985-4704-bf7e-9964b46df665-istio-podinfo\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.140053 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.139972    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-certs\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.140053 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140001    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-socket\") pod \"bdac51c2-3985-4704-bf7e-9964b46df665\" (UID: \"bdac51c2-3985-4704-bf7e-9964b46df665\") "
Sep 11 18:26:21.140053 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdac51c2-3985-4704-bf7e-9964b46df665-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:26:21.140329 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140182    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:21.140386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140338    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:21.140386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140353    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:21.140487 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-data" (OuterVolumeSpecName: "istio-data") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:21.140648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140616    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.140648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140641    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.140781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140656    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.140781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140667    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.140781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.140680    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdac51c2-3985-4704-bf7e-9964b46df665-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.142109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.142084    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdac51c2-3985-4704-bf7e-9964b46df665-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:26:21.142109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.142092    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:21.142417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.142396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-istio-token" (OuterVolumeSpecName: "istio-token") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:26:21.142417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.142408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-kube-api-access-fq4pr" (OuterVolumeSpecName: "kube-api-access-fq4pr") pod "bdac51c2-3985-4704-bf7e-9964b46df665" (UID: "bdac51c2-3985-4704-bf7e-9964b46df665"). InnerVolumeSpecName "kube-api-access-fq4pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:26:21.241865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.241804    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq4pr\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-kube-api-access-fq4pr\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.241865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.241835    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdac51c2-3985-4704-bf7e-9964b46df665-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.241865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.241846    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdac51c2-3985-4704-bf7e-9964b46df665-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.241865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.241855    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdac51c2-3985-4704-bf7e-9964b46df665-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:21.834186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.834155    2563 generic.go:356] "Generic (PLEG): container finished" podID="bdac51c2-3985-4704-bf7e-9964b46df665" containerID="dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b" exitCode=0
Sep 11 18:26:21.834626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.834215    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"
Sep 11 18:26:21.834626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.834241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" event={"ID":"bdac51c2-3985-4704-bf7e-9964b46df665","Type":"ContainerDied","Data":"dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b"}
Sep 11 18:26:21.834626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.834345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d" event={"ID":"bdac51c2-3985-4704-bf7e-9964b46df665","Type":"ContainerDied","Data":"3be03f00b8d94014d1b2d7df9e96e835ed0487561747881b6e40e1433e8effd4"}
Sep 11 18:26:21.834626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.834370    2563 scope.go:117] "RemoveContainer" containerID="dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b"
Sep 11 18:26:21.844440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.844424    2563 scope.go:117] "RemoveContainer" containerID="934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"
Sep 11 18:26:21.852671 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.852657    2563 scope.go:117] "RemoveContainer" containerID="dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b"
Sep 11 18:26:21.852920 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:26:21.852903    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b\": container with ID starting with dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b not found: ID does not exist" containerID="dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b"
Sep 11 18:26:21.852989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.852925    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b"} err="failed to get container status \"dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b\": rpc error: code = NotFound desc = could not find container \"dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b\": container with ID starting with dddd87d6bdd498a0f808ed910f5f2fa3cc18bc304092f781b79e20eba012651b not found: ID does not exist"
Sep 11 18:26:21.852989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.852940    2563 scope.go:117] "RemoveContainer" containerID="934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"
Sep 11 18:26:21.853189 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:26:21.853171    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604\": container with ID starting with 934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604 not found: ID does not exist" containerID="934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"
Sep 11 18:26:21.853229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.853195    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604"} err="failed to get container status \"934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604\": rpc error: code = NotFound desc = could not find container \"934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604\": container with ID starting with 934ba76af3f803eab01cd4bdd21adba2102a45583a83a815a7f493cf9b638604 not found: ID does not exist"
Sep 11 18:26:21.857080 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.857053    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"]
Sep 11 18:26:21.860007 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.859985    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-23f49f11-openshift-default-564dfff8bd-xr97d"]
Sep 11 18:26:21.913172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.913149    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" path="/var/lib/kubelet/pods/bdac51c2-3985-4704-bf7e-9964b46df665/volumes"
Sep 11 18:26:21.927423 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:21.927399    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw" podUID="0a0d4e95-df58-45d1-a824-a0effb83b4f1" containerName="main" containerID="cri-o://2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9" gracePeriod=45
Sep 11 18:26:22.183376 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.183346    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:26:22.249359 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249335    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-dshm\") pod \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") "
Sep 11 18:26:22.249485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249440    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-model-cache\") pod \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") "
Sep 11 18:26:22.249485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249462    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tls-certs\") pod \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") "
Sep 11 18:26:22.249587 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249486    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-home\") pod \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") "
Sep 11 18:26:22.249587 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249539    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpkgp\" (UniqueName: \"kubernetes.io/projected/0a0d4e95-df58-45d1-a824-a0effb83b4f1-kube-api-access-rpkgp\") pod \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") "
Sep 11 18:26:22.249587 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249576    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tmp-dir\") pod \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\" (UID: \"0a0d4e95-df58-45d1-a824-a0effb83b4f1\") "
Sep 11 18:26:22.249748 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249696    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a0d4e95-df58-45d1-a824-a0effb83b4f1" (UID: "0a0d4e95-df58-45d1-a824-a0effb83b4f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:22.249896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a0d4e95-df58-45d1-a824-a0effb83b4f1" (UID: "0a0d4e95-df58-45d1-a824-a0effb83b4f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:22.249978 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249960    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:22.250025 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.249983    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:22.250107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.250082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-home" (OuterVolumeSpecName: "home") pod "0a0d4e95-df58-45d1-a824-a0effb83b4f1" (UID: "0a0d4e95-df58-45d1-a824-a0effb83b4f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:22.251575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.251546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-dshm" (OuterVolumeSpecName: "dshm") pod "0a0d4e95-df58-45d1-a824-a0effb83b4f1" (UID: "0a0d4e95-df58-45d1-a824-a0effb83b4f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:26:22.251668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.251600    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a0d4e95-df58-45d1-a824-a0effb83b4f1" (UID: "0a0d4e95-df58-45d1-a824-a0effb83b4f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:26:22.251711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.251684    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0d4e95-df58-45d1-a824-a0effb83b4f1-kube-api-access-rpkgp" (OuterVolumeSpecName: "kube-api-access-rpkgp") pod "0a0d4e95-df58-45d1-a824-a0effb83b4f1" (UID: "0a0d4e95-df58-45d1-a824-a0effb83b4f1"). InnerVolumeSpecName "kube-api-access-rpkgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:26:22.350630 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.350571    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a0d4e95-df58-45d1-a824-a0effb83b4f1-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:22.350630 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.350592    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:22.350630 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.350602    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpkgp\" (UniqueName: \"kubernetes.io/projected/0a0d4e95-df58-45d1-a824-a0effb83b4f1-kube-api-access-rpkgp\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:22.350630 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.350611    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a0d4e95-df58-45d1-a824-a0effb83b4f1-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:26:22.842164 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.842132    2563 generic.go:356] "Generic (PLEG): container finished" podID="0a0d4e95-df58-45d1-a824-a0effb83b4f1" containerID="2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9" exitCode=0
Sep 11 18:26:22.842594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.842173    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"
Sep 11 18:26:22.842594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.842185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw" event={"ID":"0a0d4e95-df58-45d1-a824-a0effb83b4f1","Type":"ContainerDied","Data":"2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9"}
Sep 11 18:26:22.842594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.842215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw" event={"ID":"0a0d4e95-df58-45d1-a824-a0effb83b4f1","Type":"ContainerDied","Data":"99a06e9de64d812b66182034413f43b82afbf77f9316e261fd1f2773b47394c9"}
Sep 11 18:26:22.842594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.842231    2563 scope.go:117] "RemoveContainer" containerID="2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9"
Sep 11 18:26:22.852072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.852055    2563 scope.go:117] "RemoveContainer" containerID="2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9"
Sep 11 18:26:22.852334 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:26:22.852315    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9\": container with ID starting with 2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9 not found: ID does not exist" containerID="2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9"
Sep 11 18:26:22.852385 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.852351    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9"} err="failed to get container status \"2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9\": rpc error: code = NotFound desc = could not find container \"2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9\": container with ID starting with 2ea0d3f7909f5c8a860a93c0cd9abac43ef9449cbd0ed9c678c5e911aa00bcb9 not found: ID does not exist"
Sep 11 18:26:22.863833 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.863810    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"]
Sep 11 18:26:22.867667 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:22.867647    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-89bb6c/autores-model-89bb6c-kserve-8d98c5d67-lmxxw"]
Sep 11 18:26:23.633187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:23.633157    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:23.697674 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:23.697650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:23.767912 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:23.767881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:23.867848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:23.867823    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:23.912784 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:23.912726    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a0d4e95-df58-45d1-a824-a0effb83b4f1" path="/var/lib/kubelet/pods/0a0d4e95-df58-45d1-a824-a0effb83b4f1/volumes"
Sep 11 18:26:23.967669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:23.967644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:24.317218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:24.317180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:26.576185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:26.576150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:26.824395 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:26.824361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:27.258857 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.258828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:27.530981 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.530951    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"]
Sep 11 18:26:27.531479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531457    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a0d4e95-df58-45d1-a824-a0effb83b4f1" containerName="main"
Sep 11 18:26:27.531479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531473    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a0d4e95-df58-45d1-a824-a0effb83b4f1" containerName="main"
Sep 11 18:26:27.531581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531492    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy"
Sep 11 18:26:27.531581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531500    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy"
Sep 11 18:26:27.531581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531509    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy"
Sep 11 18:26:27.531581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531514    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy"
Sep 11 18:26:27.531704 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531604    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy"
Sep 11 18:26:27.531704 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.531616    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a0d4e95-df58-45d1-a824-a0effb83b4f1" containerName="main"
Sep 11 18:26:27.540568 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.537208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.540769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.540746    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-386ac7-openshift-default-dockercfg-ktsfn\""
Sep 11 18:26:27.546828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.546806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"]
Sep 11 18:26:27.555015 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.554960    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:27.599095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-istio-token\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-certs\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599229    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-envoy\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-credential-socket\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-data\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599381    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca3c330-b30e-457c-a227-36b10b512896-istio-podinfo\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd9z2\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-kube-api-access-sd9z2\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-socket\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.599746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.599532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca3c330-b30e-457c-a227-36b10b512896-istiod-ca-cert\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700223 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-certs\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-envoy\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-credential-socket\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-data\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700517 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca3c330-b30e-457c-a227-36b10b512896-istio-podinfo\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700517 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700441    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd9z2\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-kube-api-access-sd9z2\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700517 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-socket\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca3c330-b30e-457c-a227-36b10b512896-istiod-ca-cert\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-istio-token\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-certs\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700837 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700809    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-credential-socket\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.700911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.700889    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-socket\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.701154 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.701128    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-data\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.701314 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.701297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca3c330-b30e-457c-a227-36b10b512896-istiod-ca-cert\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.703149 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.703124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-envoy\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.703498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.703477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca3c330-b30e-457c-a227-36b10b512896-istio-podinfo\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.708192 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.708166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd9z2\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-kube-api-access-sd9z2\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.708300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.708205    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-istio-token\") pod \"e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") " pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:27.741593 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.741565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:27.850943 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:27.850880    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:28.129901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.129873    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"]
Sep 11 18:26:28.130915 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:26:28.130892    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca3c330_b30e_457c_a227_36b10b512896.slice/crio-9e8f38a79e3825aca8f3175b9eb8ded403373ebaf7dda5a34006a32c6a8a681c WatchSource:0}: Error finding container 9e8f38a79e3825aca8f3175b9eb8ded403373ebaf7dda5a34006a32c6a8a681c: Status 404 returned error can't find the container with id 9e8f38a79e3825aca8f3175b9eb8ded403373ebaf7dda5a34006a32c6a8a681c
Sep 11 18:26:28.133227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.133177    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:28.133334 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.133311    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:28.133377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.133364    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:28.878180 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.878145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" event={"ID":"9ca3c330-b30e-457c-a227-36b10b512896","Type":"ContainerStarted","Data":"ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"}
Sep 11 18:26:28.878547 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.878203    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" event={"ID":"9ca3c330-b30e-457c-a227-36b10b512896","Type":"ContainerStarted","Data":"9e8f38a79e3825aca8f3175b9eb8ded403373ebaf7dda5a34006a32c6a8a681c"}
Sep 11 18:26:28.909191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:28.909138    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podStartSLOduration=1.909121457 podStartE2EDuration="1.909121457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:26:28.904833267 +0000 UTC m=+957.498033835" watchObservedRunningTime="2026-09-11 18:26:28.909121457 +0000 UTC m=+957.502322013"
Sep 11 18:26:29.851213 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:29.851174    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:29.856833 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:29.856807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:29.884312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:29.884284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:29.885089 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:29.885067    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:30.102178 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:30.102084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:30.795851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:30.795819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:30.880001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:30.879966    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:31.697896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:31.697862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:32.580877 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:32.580846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:33.288839 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:33.288809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:33.961470 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:33.961440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:34.027369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:34.027337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:34.273401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:34.273315    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:34.445176 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:34.445143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:34.539175 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:34.539146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:48.383657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.383624    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"]
Sep 11 18:26:48.384468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.384446    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdac51c2-3985-4704-bf7e-9964b46df665" containerName="istio-proxy"
Sep 11 18:26:48.389625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.389605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.391993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.391974    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d73031\"/\"premium-w0-d73031-kserve-self-signed-certs\""
Sep 11 18:26:48.396149 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.396128    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"]
Sep 11 18:26:48.425285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.423165    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:48.497407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.497380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tmp-dir\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.497578 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.497472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-model-cache\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.497578 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.497503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-dshm\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.497578 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.497526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tls-certs\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.497703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.497630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-home\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.497703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.497651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rhfp\" (UniqueName: \"kubernetes.io/projected/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-kube-api-access-9rhfp\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598164 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-model-cache\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-dshm\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tls-certs\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-home\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rhfp\" (UniqueName: \"kubernetes.io/projected/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-kube-api-access-9rhfp\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598583 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tmp-dir\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598662 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598632    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-model-cache\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598897 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-home\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.598970 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.598918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tmp-dir\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.600794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.600778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-dshm\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.601020 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.601004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tls-certs\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.606888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.606867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rhfp\" (UniqueName: \"kubernetes.io/projected/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-kube-api-access-9rhfp\") pod \"premium-w0-d73031-kserve-9d9d9d98-jndcn\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") " pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.715239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.715181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:48.999384 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:48.999359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"]
Sep 11 18:26:49.001204 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:26:49.001170    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fd6f017_23ba_4144_bd33_e2ab511d2cb6.slice/crio-36730db426434e2a3b18d3975b9c861c3a1d24149b63f36208b6ee91998f4f8b WatchSource:0}: Error finding container 36730db426434e2a3b18d3975b9c861c3a1d24149b63f36208b6ee91998f4f8b: Status 404 returned error can't find the container with id 36730db426434e2a3b18d3975b9c861c3a1d24149b63f36208b6ee91998f4f8b
Sep 11 18:26:49.998081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:49.998047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn" event={"ID":"4fd6f017-23ba-4144-bd33-e2ab511d2cb6","Type":"ContainerStarted","Data":"c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9"}
Sep 11 18:26:49.998551 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:49.998094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn" event={"ID":"4fd6f017-23ba-4144-bd33-e2ab511d2cb6","Type":"ContainerStarted","Data":"36730db426434e2a3b18d3975b9c861c3a1d24149b63f36208b6ee91998f4f8b"}
Sep 11 18:26:50.016954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:50.016906    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn" podStartSLOduration=2.016890442 podStartE2EDuration="2.016890442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:26:50.014461533 +0000 UTC m=+978.607662094" watchObservedRunningTime="2026-09-11 18:26:50.016890442 +0000 UTC m=+978.610090999"
Sep 11 18:26:50.199247 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:50.199213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:52.009568 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:52.009504    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ca3c330-b30e-457c-a227-36b10b512896" containerID="ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283" exitCode=0
Sep 11 18:26:52.009904 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:52.009574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" event={"ID":"9ca3c330-b30e-457c-a227-36b10b512896","Type":"ContainerDied","Data":"ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"}
Sep 11 18:26:52.009946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:52.009926    2563 scope.go:117] "RemoveContainer" containerID="ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"
Sep 11 18:26:52.011015 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:52.010982    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:52.011086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:52.011056    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:52.011122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:52.011092    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:26:53.021566 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:53.021529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" event={"ID":"9ca3c330-b30e-457c-a227-36b10b512896","Type":"ContainerStarted","Data":"fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b"}
Sep 11 18:26:53.851656 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:53.851618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:53.851857 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:53.851834    2563 patch_prober.go:28] interesting pod/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 18:26:53.851908 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:53.851881    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 18:26:54.852216 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:54.852178    2563 patch_prober.go:28] interesting pod/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 18:26:54.852631 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:54.852244    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 18:26:55.851845 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:55.851812    2563 patch_prober.go:28] interesting pod/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 18:26:55.852014 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:55.851870    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 18:26:56.320777 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:56.320742    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:56.851533 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:56.851499    2563 patch_prober.go:28] interesting pod/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 18:26:56.851701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:56.851558    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 18:26:57.273016 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:57.272936    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:57.369148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:57.369114    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:57.473364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:57.473324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:57.851462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:57.851429    2563 patch_prober.go:28] interesting pod/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 18:26:57.851632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:57.851483    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 18:26:58.197982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.197907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:58.715492 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.715462    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:58.715854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.715508    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:58.727971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.727941    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:58.855817 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.855782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:58.856142 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.856123    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:58.856771 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.856748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:26:58.895818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:58.895781    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:26:59.068432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:59.068406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:26:59.584801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:26:59.584768    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:00.237404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:00.237370    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:00.829518 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:00.829487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:07.723167 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:07.723134    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:13.262622 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:13.262586    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:16.204782 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.204745    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"]
Sep 11 18:27:16.208877 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.208853    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.212169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.212142    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-12eb53\"/\"default-dockercfg-zvbbq\""
Sep 11 18:27:16.212169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.212161    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-12eb53\"/\"openshift-service-ca.crt\""
Sep 11 18:27:16.212382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.212204    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-12eb53\"/\"test-model-12eb53-kserve-self-signed-certs\""
Sep 11 18:27:16.212382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.212141    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-12eb53\"/\"kube-root-ca.crt\""
Sep 11 18:27:16.217435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.217405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"]
Sep 11 18:27:16.225773 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.225727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:16.355784 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.355739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljfdv\" (UniqueName: \"kubernetes.io/projected/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-kube-api-access-ljfdv\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.356106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.356083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-home\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.356283 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.356246    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-model-cache\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.356427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.356412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tmp-dir\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.356546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.356532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-dshm\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.356732 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.356715    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tls-certs\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-home\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-model-cache\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tmp-dir\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458313    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-dshm\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tls-certs\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458674 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljfdv\" (UniqueName: \"kubernetes.io/projected/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-kube-api-access-ljfdv\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458674 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-model-cache\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458678    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tmp-dir\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.458770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.458729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-home\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.460756 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.460733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-dshm\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.460984 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.460966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tls-certs\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.468029 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.468005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljfdv\" (UniqueName: \"kubernetes.io/projected/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-kube-api-access-ljfdv\") pod \"test-model-12eb53-kserve-55944cb585-qj7s2\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") " pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.520626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.520596    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:16.797174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:16.797146    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"]
Sep 11 18:27:16.799036 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:27:16.799008    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc286401a_f1ae_4f14_86e8_0e2813bc2ba4.slice/crio-d6b96e6ca6ff4409ecb1d75ca121aa0ebba5cc97bd18b409ee52c7356b7b7f9d WatchSource:0}: Error finding container d6b96e6ca6ff4409ecb1d75ca121aa0ebba5cc97bd18b409ee52c7356b7b7f9d: Status 404 returned error can't find the container with id d6b96e6ca6ff4409ecb1d75ca121aa0ebba5cc97bd18b409ee52c7356b7b7f9d
Sep 11 18:27:17.153047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:17.152962    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2" event={"ID":"c286401a-f1ae-4f14-86e8-0e2813bc2ba4","Type":"ContainerStarted","Data":"680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649"}
Sep 11 18:27:17.153047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:17.153007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2" event={"ID":"c286401a-f1ae-4f14-86e8-0e2813bc2ba4","Type":"ContainerStarted","Data":"d6b96e6ca6ff4409ecb1d75ca121aa0ebba5cc97bd18b409ee52c7356b7b7f9d"}
Sep 11 18:27:17.171035 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:17.170988    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2" podStartSLOduration=1.170972818 podStartE2EDuration="1.170972818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:27:17.169863508 +0000 UTC m=+1005.763064077" watchObservedRunningTime="2026-09-11 18:27:17.170972818 +0000 UTC m=+1005.764173685"
Sep 11 18:27:19.143138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:19.143107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:19.258508 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:19.258477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:19.397809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:19.397728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:24.827575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:24.827523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:26.521135 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:26.521098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:26.521519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:26.521149    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:26.533508 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:26.533483    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:27.220973 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:27.220942    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:27:29.529348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:29.529321    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:29.995794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:29.995705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:30.712496 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:30.712462    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:36.013322 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:36.013292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:36.079608 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:36.079555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:36.583722 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:36.583691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:36.700374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:36.700338    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:36.782625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:36.782595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:36.883005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:36.882929    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:37.475172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:37.475120    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:37.571476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:37.571450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:37.675297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:37.675253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:39.107437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:39.107405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:41.452274 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:41.452231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:42.486596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:42.486553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:43.202877 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:43.202837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:44.194729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:44.194694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:44.877157 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:44.877121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:45.906820 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:45.906786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:46.003221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:46.003190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:46.586496 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:46.586463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:46.694072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:46.694039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.346444 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.346412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.416107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.416075    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.475643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.475612    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.575910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.575882    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.677624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.677521    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.881918 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.881876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:52.999562 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:52.999486    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:53.627412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:53.627378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:53.727789 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:53.727760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:53.854901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:53.854868    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:54.151813 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:54.151777    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:54.308048 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:54.308018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:54.913797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:54.913761    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:55.694717 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:55.694685    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:56.588978 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:56.588944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:57.962456 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:57.962417    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:27:59.894477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:27:59.894438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:00.510614 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:00.510583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:00.607909 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:00.607877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:00.708273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:00.708237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:01.408935 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:01.408903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:02.380569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:02.380533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:03.196589 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:03.196503    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:04.112027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:04.111997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:04.213693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:04.213663    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:04.319250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:04.319218    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:04.580275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:04.580243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:04.657102 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:04.657078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:05.104323 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:05.104287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:05.344388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:05.344352    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"]
Sep 11 18:28:05.344763 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:05.344607    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy" containerID="cri-o://fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b" gracePeriod=30
Sep 11 18:28:05.367828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:05.367757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:05.785332 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:05.785290    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:06.131205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:06.131128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:06.311734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:06.311701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:07.307466 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:07.307439    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:07.441240 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:07.441205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:07.585810 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:07.585733    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:10.605236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.605212    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:28:10.670885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670806    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-certs\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.670885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670844    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-data\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.670885 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670871    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-socket\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca3c330-b30e-457c-a227-36b10b512896-istio-podinfo\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca3c330-b30e-457c-a227-36b10b512896-istiod-ca-cert\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd9z2\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-kube-api-access-sd9z2\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670975    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-istio-token\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.670999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-envoy\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.671077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-credential-socket\") pod \"9ca3c330-b30e-457c-a227-36b10b512896\" (UID: \"9ca3c330-b30e-457c-a227-36b10b512896\") "
Sep 11 18:28:10.671720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.671682    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:28:10.671948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.671911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:28:10.672037 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.671966    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:28:10.672204 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.671131    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:28:10.673615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.672733    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.673615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.672762    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.673615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.672776    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.673615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.672800    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.673950 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.673792    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca3c330-b30e-457c-a227-36b10b512896-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:28:10.674410 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.674386    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ca3c330-b30e-457c-a227-36b10b512896-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:28:10.674410 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.674396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:28:10.674575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.674422    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:28:10.674575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.674460    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-kube-api-access-sd9z2" (OuterVolumeSpecName: "kube-api-access-sd9z2") pod "9ca3c330-b30e-457c-a227-36b10b512896" (UID: "9ca3c330-b30e-457c-a227-36b10b512896"). InnerVolumeSpecName "kube-api-access-sd9z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:28:10.773835 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.773799    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ca3c330-b30e-457c-a227-36b10b512896-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.773835 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.773831    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ca3c330-b30e-457c-a227-36b10b512896-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.774019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.773842    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd9z2\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-kube-api-access-sd9z2\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.774019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.773851    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ca3c330-b30e-457c-a227-36b10b512896-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:10.774019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:10.773860    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ca3c330-b30e-457c-a227-36b10b512896-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:28:11.454992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.454960    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ca3c330-b30e-457c-a227-36b10b512896" containerID="fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b" exitCode=0
Sep 11 18:28:11.455130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.455022    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"
Sep 11 18:28:11.455130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.455045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" event={"ID":"9ca3c330-b30e-457c-a227-36b10b512896","Type":"ContainerDied","Data":"fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b"}
Sep 11 18:28:11.455130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.455111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k" event={"ID":"9ca3c330-b30e-457c-a227-36b10b512896","Type":"ContainerDied","Data":"9e8f38a79e3825aca8f3175b9eb8ded403373ebaf7dda5a34006a32c6a8a681c"}
Sep 11 18:28:11.455251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.455142    2563 scope.go:117] "RemoveContainer" containerID="fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b"
Sep 11 18:28:11.465172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.465004    2563 scope.go:117] "RemoveContainer" containerID="ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"
Sep 11 18:28:11.473865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.473846    2563 scope.go:117] "RemoveContainer" containerID="fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b"
Sep 11 18:28:11.474136 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:28:11.474116    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b\": container with ID starting with fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b not found: ID does not exist" containerID="fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b"
Sep 11 18:28:11.474202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.474148    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b"} err="failed to get container status \"fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b\": rpc error: code = NotFound desc = could not find container \"fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b\": container with ID starting with fb8df3558a3f279c9645b5c36aaaa039e6ab6c33fac0b604d2c0b7e44ceeca8b not found: ID does not exist"
Sep 11 18:28:11.474202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.474172    2563 scope.go:117] "RemoveContainer" containerID="ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"
Sep 11 18:28:11.474451 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:28:11.474434    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283\": container with ID starting with ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283 not found: ID does not exist" containerID="ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"
Sep 11 18:28:11.474492 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.474457    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283"} err="failed to get container status \"ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283\": rpc error: code = NotFound desc = could not find container \"ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283\": container with ID starting with ea85fbfa3b567f24cbd09bb83fd98b3d1c62b6e4409c2b990ca945b6b5ef2283 not found: ID does not exist"
Sep 11 18:28:11.477760 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.477734    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"]
Sep 11 18:28:11.482205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.482182    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-386ac7-openshift-default-8c45cc8b8-rsz8k"]
Sep 11 18:28:11.913237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:11.913208    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca3c330-b30e-457c-a227-36b10b512896" path="/var/lib/kubelet/pods/9ca3c330-b30e-457c-a227-36b10b512896/volumes"
Sep 11 18:28:26.223156 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.223087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:26.507112 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.507031    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"]
Sep 11 18:28:26.507633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.507613    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy"
Sep 11 18:28:26.507633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.507631    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy"
Sep 11 18:28:26.507775 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.507643    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy"
Sep 11 18:28:26.507775 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.507649    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy"
Sep 11 18:28:26.507775 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.507726    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy"
Sep 11 18:28:26.512514 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.512496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.514909 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.514886    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"e2e-conflict-a-c17caa91-kserve-self-signed-certs\""
Sep 11 18:28:26.515603 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.515582    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"kube-root-ca.crt\""
Sep 11 18:28:26.515709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.515655    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"openshift-service-ca.crt\""
Sep 11 18:28:26.515709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.515674    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"default-dockercfg-j5lh6\""
Sep 11 18:28:26.522235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.522213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"]
Sep 11 18:28:26.535067 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.535033    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:26.612735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.612708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c52qh\" (UniqueName: \"kubernetes.io/projected/0a70ce38-609c-4eae-b19a-08a17d738824-kube-api-access-c52qh\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.612846 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.612753    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-dshm\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.612846 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.612806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a70ce38-609c-4eae-b19a-08a17d738824-tls-certs\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.612846 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.612834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-home\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.612951 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.612917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-tmp-dir\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.612951 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.612944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-model-cache\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714006 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.713973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-model-cache\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714153 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c52qh\" (UniqueName: \"kubernetes.io/projected/0a70ce38-609c-4eae-b19a-08a17d738824-kube-api-access-c52qh\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714153 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-dshm\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714153 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a70ce38-609c-4eae-b19a-08a17d738824-tls-certs\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714153 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-home\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714428 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-tmp-dir\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714459    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-model-cache\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714634 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-tmp-dir\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.714756 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.714655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-home\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.716407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.716372    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-dshm\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.716684 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.716665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a70ce38-609c-4eae-b19a-08a17d738824-tls-certs\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.722124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.722103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c52qh\" (UniqueName: \"kubernetes.io/projected/0a70ce38-609c-4eae-b19a-08a17d738824-kube-api-access-c52qh\") pod \"e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") " pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:26.822766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:26.822745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:27.082336 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:27.082237    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:27.100134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:27.100110    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"]
Sep 11 18:28:27.101054 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:28:27.101025    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a70ce38_609c_4eae_b19a_08a17d738824.slice/crio-2c0fe1fdaf26d60736b49c52ed1d0079a93b664fbd99f10f3ab2daced03c677a WatchSource:0}: Error finding container 2c0fe1fdaf26d60736b49c52ed1d0079a93b664fbd99f10f3ab2daced03c677a: Status 404 returned error can't find the container with id 2c0fe1fdaf26d60736b49c52ed1d0079a93b664fbd99f10f3ab2daced03c677a
Sep 11 18:28:27.545102 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:27.545072    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs" event={"ID":"0a70ce38-609c-4eae-b19a-08a17d738824","Type":"ContainerStarted","Data":"3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc"}
Sep 11 18:28:27.545475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:27.545134    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs" event={"ID":"0a70ce38-609c-4eae-b19a-08a17d738824","Type":"ContainerStarted","Data":"2c0fe1fdaf26d60736b49c52ed1d0079a93b664fbd99f10f3ab2daced03c677a"}
Sep 11 18:28:27.564947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:27.564905    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs" podStartSLOduration=1.564889667 podStartE2EDuration="1.564889667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:28:27.564126287 +0000 UTC m=+1076.157326847" watchObservedRunningTime="2026-09-11 18:28:27.564889667 +0000 UTC m=+1076.158090224"
Sep 11 18:28:27.912615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:27.912528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:28.622400 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:28.622366    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:29.386314 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:29.386285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:30.076087 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:30.076056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:30.582678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:30.582648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:31.217747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:31.217717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:31.311636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:31.311583    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:31.630625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:31.630596    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:36.823098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:36.823066    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:36.823796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:36.823118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:36.836509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:36.836478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:37.612341 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:37.612313    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:28:37.680351 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:37.680310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:37.782122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:37.782091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:38.998124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:38.998088    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:39.757269 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:39.757229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:40.488601 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:40.488565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:41.416303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:41.416270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:42.138250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.138205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:42.348008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.347974    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"]
Sep 11 18:28:42.348539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.348524    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca3c330-b30e-457c-a227-36b10b512896" containerName="istio-proxy"
Sep 11 18:28:42.353049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.353031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.356193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.356177    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-d33a60\"/\"default-dockercfg-z6mb8\""
Sep 11 18:28:42.356933 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.356915    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-d33a60\"/\"kube-root-ca.crt\""
Sep 11 18:28:42.357010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.356957    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-d33a60\"/\"openshift-service-ca.crt\""
Sep 11 18:28:42.357010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.356993    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-d33a60\"/\"noten-model-d33a60-kserve-self-signed-certs\""
Sep 11 18:28:42.372023 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.371997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"]
Sep 11 18:28:42.382856 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.382820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:42.464392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.464339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-model-cache\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.464392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.464380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h42z7\" (UniqueName: \"kubernetes.io/projected/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-kube-api-access-h42z7\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.464557 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.464488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tmp-dir\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.464557 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.464546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tls-certs\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.464664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.464600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-dshm\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.464664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.464643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-home\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565355 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tmp-dir\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565469 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tls-certs\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565469 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-dshm\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-home\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-model-cache\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h42z7\" (UniqueName: \"kubernetes.io/projected/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-kube-api-access-h42z7\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tmp-dir\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565904 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-home\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.565904 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.565895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-model-cache\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.567772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.567755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-dshm\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.568071 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.568055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tls-certs\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.573898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.573877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h42z7\" (UniqueName: \"kubernetes.io/projected/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-kube-api-access-h42z7\") pod \"noten-model-d33a60-kserve-cbc898bc5-nssk8\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") " pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.662132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.662102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:42.940085 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:42.940053    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"]
Sep 11 18:28:42.948805 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:28:42.948777    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode32a2fb2_a40e_4dd5_81ad_567e36e9d4ed.slice/crio-d7749abdae2838802b4cf4717d3aa9f4224b7fedf806cd20a609364441edb0f4 WatchSource:0}: Error finding container d7749abdae2838802b4cf4717d3aa9f4224b7fedf806cd20a609364441edb0f4: Status 404 returned error can't find the container with id d7749abdae2838802b4cf4717d3aa9f4224b7fedf806cd20a609364441edb0f4
Sep 11 18:28:43.022479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:43.022451    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:43.636653 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:43.636614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8" event={"ID":"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed","Type":"ContainerStarted","Data":"2213203cebd24c278593d75a553047d32b880c9ea3792dded32fb5a705789954"}
Sep 11 18:28:43.637229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:43.636660    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8" event={"ID":"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed","Type":"ContainerStarted","Data":"d7749abdae2838802b4cf4717d3aa9f4224b7fedf806cd20a609364441edb0f4"}
Sep 11 18:28:43.656800 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:43.656756    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8" podStartSLOduration=1.6567419559999998 podStartE2EDuration="1.656741956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:28:43.65468081 +0000 UTC m=+1092.247881370" watchObservedRunningTime="2026-09-11 18:28:43.656741956 +0000 UTC m=+1092.249942542"
Sep 11 18:28:48.723902 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:48.723867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:48.825805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:48.825768    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:48.926491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:48.926463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:49.026401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:49.026362    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:49.328966 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:49.328883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:49.400210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:49.400186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:50.087428 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:50.087399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:51.002957 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:51.002923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:51.703797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:51.703767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:52.459633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:52.459598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:52.662818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:52.662787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:52.662986 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:52.662865    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:52.675019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:52.674998    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:52.700323 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:52.700296    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:28:55.259097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:55.259052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:55.692382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:55.692339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:55.792093 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:55.792058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:55.964992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:55.964906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:56.332250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:56.332216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:56.448490 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:56.448463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:28:56.533989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:28:56.533965    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:02.685611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:02.685576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:02.794329 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:02.794248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:02.898828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:02.898795    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:03.048914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:03.048885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:09.208283 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:09.208230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:09.309055 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:09.309032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:10.102137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:10.102105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:10.437175 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:10.437084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:10.757229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:10.757146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:12.634080 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:12.634047    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:13.519953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:13.519914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:13.639315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:13.639286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:14.478722 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:14.478692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:15.392676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:15.392636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:15.506189 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:15.506157    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:15.593435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:15.593406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:15.700218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:15.700151    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:15.996651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:15.996545    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:16.203893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:16.203858    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:16.940308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:16.940274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:17.234386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:17.234308    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:17.337632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:17.337601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:17.501122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:17.501042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:17.763643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:17.763557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:17.861506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:17.861474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:20.670678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:20.670647    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"]
Sep 11 18:29:20.776942 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:20.776914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:35.690666 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:35.690608    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8" podUID="e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" containerName="main" containerID="cri-o://2213203cebd24c278593d75a553047d32b880c9ea3792dded32fb5a705789954" gracePeriod=45
Sep 11 18:29:35.924233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:35.924204    2563 generic.go:356] "Generic (PLEG): container finished" podID="e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" containerID="2213203cebd24c278593d75a553047d32b880c9ea3792dded32fb5a705789954" exitCode=0
Sep 11 18:29:35.924388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:35.924271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8" event={"ID":"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed","Type":"ContainerDied","Data":"2213203cebd24c278593d75a553047d32b880c9ea3792dded32fb5a705789954"}
Sep 11 18:29:35.959672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:35.959602    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:29:36.050812 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.050782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tls-certs\") pod \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") "
Sep 11 18:29:36.050999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.050874    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-dshm\") pod \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") "
Sep 11 18:29:36.050999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.050900    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-model-cache\") pod \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") "
Sep 11 18:29:36.050999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.050918    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h42z7\" (UniqueName: \"kubernetes.io/projected/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-kube-api-access-h42z7\") pod \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") "
Sep 11 18:29:36.050999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.050952    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-home\") pod \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") "
Sep 11 18:29:36.051215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.051008    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tmp-dir\") pod \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\" (UID: \"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed\") "
Sep 11 18:29:36.051292 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.051251    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" (UID: "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:36.051357 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.051307    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-home" (OuterVolumeSpecName: "home") pod "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" (UID: "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:36.051434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.051399    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" (UID: "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:36.051434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.051426    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:36.051521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.051445    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:36.053161 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.053138    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-dshm" (OuterVolumeSpecName: "dshm") pod "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" (UID: "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:36.053161 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.053154    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" (UID: "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:29:36.053304 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.053224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-kube-api-access-h42z7" (OuterVolumeSpecName: "kube-api-access-h42z7") pod "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" (UID: "e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed"). InnerVolumeSpecName "kube-api-access-h42z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:29:36.152537 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.152516    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:36.152537 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.152536    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h42z7\" (UniqueName: \"kubernetes.io/projected/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-kube-api-access-h42z7\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:36.152679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.152547    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:36.152679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.152557    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:36.863159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.863127    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:36.931305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.931278    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"
Sep 11 18:29:36.931305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.931291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8" event={"ID":"e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed","Type":"ContainerDied","Data":"d7749abdae2838802b4cf4717d3aa9f4224b7fedf806cd20a609364441edb0f4"}
Sep 11 18:29:36.931518 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.931340    2563 scope.go:117] "RemoveContainer" containerID="2213203cebd24c278593d75a553047d32b880c9ea3792dded32fb5a705789954"
Sep 11 18:29:36.955935 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.955906    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"]
Sep 11 18:29:36.961059 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:36.961038    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-d33a60/noten-model-d33a60-kserve-cbc898bc5-nssk8"]
Sep 11 18:29:37.689583 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:37.689550    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:37.913245 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:37.913216    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" path="/var/lib/kubelet/pods/e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed/volumes"
Sep 11 18:29:38.207794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:38.207754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:38.509574 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:38.509496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:39.176873 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:39.176841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:39.602115 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:39.602089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:39.709075 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:39.709045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:40.112188 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:40.112150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:40.479359 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:40.479280    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"]
Sep 11 18:29:40.502880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:40.502847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:40.966347 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:40.966312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:41.890482 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:41.890444    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:42.703764 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:42.703727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:43.402595 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:43.402567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:43.503341 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:43.503307    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:43.743433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:43.743346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:44.065247 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:44.065209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:44.293536 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:44.293504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:44.688220 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:44.688188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:45.252159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:45.252115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:45.986733 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:45.986705    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:46.903718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:46.903686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:47.691613 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:47.691579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:47.796446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:47.796393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:48.544919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:48.544885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:49.472790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:49.472758    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:50.181362 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:50.181319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:51.002049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.002009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:51.629335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.629296    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:51.652712 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.652683    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"]
Sep 11 18:29:51.653002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.652961    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp" podUID="2e43710c-a78c-461f-808e-49951e027dd7" containerName="payload-pre-processing" containerID="cri-o://99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c" gracePeriod=30
Sep 11 18:29:51.729370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.729343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:51.832793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.832764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:51.917249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.917214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:51.922220 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:51.922189    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:29:52.008389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.008359    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e43710c-a78c-461f-808e-49951e027dd7-plugins-config-volume\") pod \"2e43710c-a78c-461f-808e-49951e027dd7\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") "
Sep 11 18:29:52.008714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.008515    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vg87\" (UniqueName: \"kubernetes.io/projected/2e43710c-a78c-461f-808e-49951e027dd7-kube-api-access-9vg87\") pod \"2e43710c-a78c-461f-808e-49951e027dd7\" (UID: \"2e43710c-a78c-461f-808e-49951e027dd7\") "
Sep 11 18:29:52.008714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.008678    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e43710c-a78c-461f-808e-49951e027dd7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e43710c-a78c-461f-808e-49951e027dd7" (UID: "2e43710c-a78c-461f-808e-49951e027dd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:29:52.009551 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.009528    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e43710c-a78c-461f-808e-49951e027dd7-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:52.010843 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.010821    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e43710c-a78c-461f-808e-49951e027dd7-kube-api-access-9vg87" (OuterVolumeSpecName: "kube-api-access-9vg87") pod "2e43710c-a78c-461f-808e-49951e027dd7" (UID: "2e43710c-a78c-461f-808e-49951e027dd7"). InnerVolumeSpecName "kube-api-access-9vg87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:29:52.018329 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.018303    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e43710c-a78c-461f-808e-49951e027dd7" containerID="99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c" exitCode=0
Sep 11 18:29:52.018450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.018338    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"
Sep 11 18:29:52.018450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.018379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp" event={"ID":"2e43710c-a78c-461f-808e-49951e027dd7","Type":"ContainerDied","Data":"99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c"}
Sep 11 18:29:52.018450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.018432    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp" event={"ID":"2e43710c-a78c-461f-808e-49951e027dd7","Type":"ContainerDied","Data":"a24eb9e24e2f743b4b1dee4c75c5a0ee84f025ccc17ec5d1b90c16531425c1dc"}
Sep 11 18:29:52.018579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.018456    2563 scope.go:117] "RemoveContainer" containerID="99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c"
Sep 11 18:29:52.030279 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.030235    2563 scope.go:117] "RemoveContainer" containerID="99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c"
Sep 11 18:29:52.030676 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:29:52.030654    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c\": container with ID starting with 99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c not found: ID does not exist" containerID="99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c"
Sep 11 18:29:52.030778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.030689    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c"} err="failed to get container status \"99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c\": rpc error: code = NotFound desc = could not find container \"99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c\": container with ID starting with 99302d328d331ecc571afe2920ecefab1974f77b90d965db0c747556f09c953c not found: ID does not exist"
Sep 11 18:29:52.050001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.049979    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"]
Sep 11 18:29:52.066241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.066216    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-f63893-79bff95d66-9qcqp"]
Sep 11 18:29:52.101474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.101449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:52.110999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.110974    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vg87\" (UniqueName: \"kubernetes.io/projected/2e43710c-a78c-461f-808e-49951e027dd7-kube-api-access-9vg87\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:52.201369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.201288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:52.301124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.301082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:52.380342 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.380301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:52.621650 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.621622    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:52.720292 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.720229    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:52.797737 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.797706    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"]
Sep 11 18:29:52.798155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.798141    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" containerName="main"
Sep 11 18:29:52.798155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.798156    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" containerName="main"
Sep 11 18:29:52.798272 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.798196    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e43710c-a78c-461f-808e-49951e027dd7" containerName="payload-pre-processing"
Sep 11 18:29:52.798272 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.798202    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e43710c-a78c-461f-808e-49951e027dd7" containerName="payload-pre-processing"
Sep 11 18:29:52.798339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.798288    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e32a2fb2-a40e-4dd5-81ad-567e36e9d4ed" containerName="main"
Sep 11 18:29:52.798339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.798298    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e43710c-a78c-461f-808e-49951e027dd7" containerName="payload-pre-processing"
Sep 11 18:29:52.802445 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.802429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:52.804611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.804581    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-6c3180-dockercfg-jk4tr\""
Sep 11 18:29:52.804611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.804611    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-6c3180\""
Sep 11 18:29:52.821327 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.821302    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"]
Sep 11 18:29:52.922886 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.922804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f77b50-fdee-44b3-b715-a291cf34720f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:52.923028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:52.922913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsq95\" (UniqueName: \"kubernetes.io/projected/39f77b50-fdee-44b3-b715-a291cf34720f-kube-api-access-rsq95\") pod \"payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:53.023622 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.023590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsq95\" (UniqueName: \"kubernetes.io/projected/39f77b50-fdee-44b3-b715-a291cf34720f-kube-api-access-rsq95\") pod \"payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:53.024091 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.023925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f77b50-fdee-44b3-b715-a291cf34720f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:53.024690 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.024664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f77b50-fdee-44b3-b715-a291cf34720f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:53.031411 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.031390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsq95\" (UniqueName: \"kubernetes.io/projected/39f77b50-fdee-44b3-b715-a291cf34720f-kube-api-access-rsq95\") pod \"payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:53.125210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.125183    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:53.182172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.182024    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:53.398194 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.398172    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"]
Sep 11 18:29:53.400481 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:29:53.400458    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39f77b50_fdee_44b3_b715_a291cf34720f.slice/crio-2af183f8ea11dbde77bbe7f4c23b236fb66d50c6f1045491cd3296933b62e4d9 WatchSource:0}: Error finding container 2af183f8ea11dbde77bbe7f4c23b236fb66d50c6f1045491cd3296933b62e4d9: Status 404 returned error can't find the container with id 2af183f8ea11dbde77bbe7f4c23b236fb66d50c6f1045491cd3296933b62e4d9
Sep 11 18:29:53.883695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.883656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:53.915008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:53.914973    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e43710c-a78c-461f-808e-49951e027dd7" path="/var/lib/kubelet/pods/2e43710c-a78c-461f-808e-49951e027dd7/volumes"
Sep 11 18:29:54.035225 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:54.035194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56" event={"ID":"39f77b50-fdee-44b3-b715-a291cf34720f","Type":"ContainerStarted","Data":"f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b"}
Sep 11 18:29:54.035676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:54.035245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56" event={"ID":"39f77b50-fdee-44b3-b715-a291cf34720f","Type":"ContainerStarted","Data":"2af183f8ea11dbde77bbe7f4c23b236fb66d50c6f1045491cd3296933b62e4d9"}
Sep 11 18:29:54.035676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:54.035292    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:29:54.053911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:54.053867    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56" podStartSLOduration=2.05384804 podStartE2EDuration="2.05384804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:29:54.051628503 +0000 UTC m=+1162.644829060" watchObservedRunningTime="2026-09-11 18:29:54.05384804 +0000 UTC m=+1162.647048597"
Sep 11 18:29:54.592842 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:54.592811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:55.406570 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.406543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:55.499386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.499346    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2" podUID="c286401a-f1ae-4f14-86e8-0e2813bc2ba4" containerName="main" containerID="cri-o://680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649" gracePeriod=45
Sep 11 18:29:55.756227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.756203    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:29:55.848470 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tmp-dir\") pod \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") "
Sep 11 18:29:55.848581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848567    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tls-certs\") pod \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") "
Sep 11 18:29:55.848639 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-home\") pod \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") "
Sep 11 18:29:55.848639 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848628    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljfdv\" (UniqueName: \"kubernetes.io/projected/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-kube-api-access-ljfdv\") pod \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") "
Sep 11 18:29:55.848745 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-dshm\") pod \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") "
Sep 11 18:29:55.848745 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848720    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c286401a-f1ae-4f14-86e8-0e2813bc2ba4" (UID: "c286401a-f1ae-4f14-86e8-0e2813bc2ba4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:55.848745 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848732    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-model-cache\") pod \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\" (UID: \"c286401a-f1ae-4f14-86e8-0e2813bc2ba4\") "
Sep 11 18:29:55.848963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848930    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-home" (OuterVolumeSpecName: "home") pod "c286401a-f1ae-4f14-86e8-0e2813bc2ba4" (UID: "c286401a-f1ae-4f14-86e8-0e2813bc2ba4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:55.849040 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.848959    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c286401a-f1ae-4f14-86e8-0e2813bc2ba4" (UID: "c286401a-f1ae-4f14-86e8-0e2813bc2ba4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:55.849227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.849186    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:55.849227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.849212    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:55.849227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.849224    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:55.850802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.850778    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-dshm" (OuterVolumeSpecName: "dshm") pod "c286401a-f1ae-4f14-86e8-0e2813bc2ba4" (UID: "c286401a-f1ae-4f14-86e8-0e2813bc2ba4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:29:55.850890 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.850875    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c286401a-f1ae-4f14-86e8-0e2813bc2ba4" (UID: "c286401a-f1ae-4f14-86e8-0e2813bc2ba4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:29:55.850931 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.850892    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-kube-api-access-ljfdv" (OuterVolumeSpecName: "kube-api-access-ljfdv") pod "c286401a-f1ae-4f14-86e8-0e2813bc2ba4" (UID: "c286401a-f1ae-4f14-86e8-0e2813bc2ba4"). InnerVolumeSpecName "kube-api-access-ljfdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:29:55.950433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.950413    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:55.950433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.950436    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljfdv\" (UniqueName: \"kubernetes.io/projected/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-kube-api-access-ljfdv\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:55.950573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:55.950452    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c286401a-f1ae-4f14-86e8-0e2813bc2ba4-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:29:56.047794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.047763    2563 generic.go:356] "Generic (PLEG): container finished" podID="c286401a-f1ae-4f14-86e8-0e2813bc2ba4" containerID="680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649" exitCode=0
Sep 11 18:29:56.047969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.047808    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"
Sep 11 18:29:56.047969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.047853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2" event={"ID":"c286401a-f1ae-4f14-86e8-0e2813bc2ba4","Type":"ContainerDied","Data":"680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649"}
Sep 11 18:29:56.047969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.047904    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2" event={"ID":"c286401a-f1ae-4f14-86e8-0e2813bc2ba4","Type":"ContainerDied","Data":"d6b96e6ca6ff4409ecb1d75ca121aa0ebba5cc97bd18b409ee52c7356b7b7f9d"}
Sep 11 18:29:56.047969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.047928    2563 scope.go:117] "RemoveContainer" containerID="680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649"
Sep 11 18:29:56.057065 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.057048    2563 scope.go:117] "RemoveContainer" containerID="680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649"
Sep 11 18:29:56.057352 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:29:56.057326    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649\": container with ID starting with 680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649 not found: ID does not exist" containerID="680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649"
Sep 11 18:29:56.057415 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.057365    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649"} err="failed to get container status \"680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649\": rpc error: code = NotFound desc = could not find container \"680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649\": container with ID starting with 680f15bb34e7406e117d8f0608bf34254b1a853477ff066a7d4d583ff27fc649 not found: ID does not exist"
Sep 11 18:29:56.069324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.069277    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"]
Sep 11 18:29:56.078755 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.078733    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-12eb53/test-model-12eb53-kserve-55944cb585-qj7s2"]
Sep 11 18:29:56.106312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.106289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:56.987556 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:56.987517    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:29:57.913245 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:57.913210    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c286401a-f1ae-4f14-86e8-0e2813bc2ba4" path="/var/lib/kubelet/pods/c286401a-f1ae-4f14-86e8-0e2813bc2ba4/volumes"
Sep 11 18:29:58.873709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:29:58.873677    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:00.144047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.144010    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"]
Sep 11 18:30:00.144545 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.144528    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c286401a-f1ae-4f14-86e8-0e2813bc2ba4" containerName="main"
Sep 11 18:30:00.144545 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.144546    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c286401a-f1ae-4f14-86e8-0e2813bc2ba4" containerName="main"
Sep 11 18:30:00.144687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.144666    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c286401a-f1ae-4f14-86e8-0e2813bc2ba4" containerName="main"
Sep 11 18:30:00.147664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.147643    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"]
Sep 11 18:30:00.147794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.147771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"
Sep 11 18:30:00.150044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.150024    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t9kdx\""
Sep 11 18:30:00.150931 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.150914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"
Sep 11 18:30:00.161128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.161109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"
Sep 11 18:30:00.161475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.161460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"]
Sep 11 18:30:00.163886 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.163862    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"]
Sep 11 18:30:00.172889 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.172871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"
Sep 11 18:30:00.302786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.302758    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:00.403273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.403180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:00.444363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.444334    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"]
Sep 11 18:30:00.445494 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:30:00.445469    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4dcb34_e6bd_48da_a121_dd77e4058a97.slice/crio-e98245f1390776c4edd6ca26466c880e80c01bcdc1abded03039c217c47d8d47 WatchSource:0}: Error finding container e98245f1390776c4edd6ca26466c880e80c01bcdc1abded03039c217c47d8d47: Status 404 returned error can't find the container with id e98245f1390776c4edd6ca26466c880e80c01bcdc1abded03039c217c47d8d47
Sep 11 18:30:00.463725 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.463705    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"]
Sep 11 18:30:00.465935 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:30:00.465900    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a0b8e7_bb2d_428a_876a_fbbefdf8ba06.slice/crio-8f0feb7a5e7b14843cd93a7de1aa6fde0f753f5b40cbe9f9b46f94c3530606a0 WatchSource:0}: Error finding container 8f0feb7a5e7b14843cd93a7de1aa6fde0f753f5b40cbe9f9b46f94c3530606a0: Status 404 returned error can't find the container with id 8f0feb7a5e7b14843cd93a7de1aa6fde0f753f5b40cbe9f9b46f94c3530606a0
Sep 11 18:30:00.561696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.561662    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"]
Sep 11 18:30:00.561925 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.561898    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy" containerID="cri-o://0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463" gracePeriod=30
Sep 11 18:30:00.576116 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:00.576089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:01.002847 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.002763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:01.077129 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.077093    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn" event={"ID":"7e4dcb34-e6bd-48da-a121-dd77e4058a97","Type":"ContainerStarted","Data":"521deb98263959df9204ccc7351391ca90e0baf89747c7b6636b7804210c6414"}
Sep 11 18:30:01.077352 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.077165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn" event={"ID":"7e4dcb34-e6bd-48da-a121-dd77e4058a97","Type":"ContainerStarted","Data":"e98245f1390776c4edd6ca26466c880e80c01bcdc1abded03039c217c47d8d47"}
Sep 11 18:30:01.078669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.078642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k" event={"ID":"89a0b8e7-bb2d-428a-876a-fbbefdf8ba06","Type":"ContainerStarted","Data":"3a9d89e56c4e5679da81ce766eb1f046a436073191a7f059d44cc92a163fb28b"}
Sep 11 18:30:01.078766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.078699    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k" event={"ID":"89a0b8e7-bb2d-428a-876a-fbbefdf8ba06","Type":"ContainerStarted","Data":"8f0feb7a5e7b14843cd93a7de1aa6fde0f753f5b40cbe9f9b46f94c3530606a0"}
Sep 11 18:30:01.094931 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.094888    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn" podStartSLOduration=1.094874944 podStartE2EDuration="1.094874944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:01.092561362 +0000 UTC m=+1169.685761920" watchObservedRunningTime="2026-09-11 18:30:01.094874944 +0000 UTC m=+1169.688075501"
Sep 11 18:30:01.112392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.112345    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k" podStartSLOduration=1.11233051 podStartE2EDuration="1.11233051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:01.110003803 +0000 UTC m=+1169.703204361" watchObservedRunningTime="2026-09-11 18:30:01.11233051 +0000 UTC m=+1169.705531066"
Sep 11 18:30:01.709459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:01.709373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:02.085293 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:02.085244    2563 generic.go:356] "Generic (PLEG): container finished" podID="7e4dcb34-e6bd-48da-a121-dd77e4058a97" containerID="521deb98263959df9204ccc7351391ca90e0baf89747c7b6636b7804210c6414" exitCode=0
Sep 11 18:30:02.085494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:02.085325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn" event={"ID":"7e4dcb34-e6bd-48da-a121-dd77e4058a97","Type":"ContainerDied","Data":"521deb98263959df9204ccc7351391ca90e0baf89747c7b6636b7804210c6414"}
Sep 11 18:30:02.086938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:02.086911    2563 generic.go:356] "Generic (PLEG): container finished" podID="89a0b8e7-bb2d-428a-876a-fbbefdf8ba06" containerID="3a9d89e56c4e5679da81ce766eb1f046a436073191a7f059d44cc92a163fb28b" exitCode=0
Sep 11 18:30:02.087046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:02.086986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k" event={"ID":"89a0b8e7-bb2d-428a-876a-fbbefdf8ba06","Type":"ContainerDied","Data":"3a9d89e56c4e5679da81ce766eb1f046a436073191a7f059d44cc92a163fb28b"}
Sep 11 18:30:02.589300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:02.589274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:03.256071 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:03.256048    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"
Sep 11 18:30:03.259248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:03.259227    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"
Sep 11 18:30:03.372920 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:03.372847    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:04.099937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.099911    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"
Sep 11 18:30:04.100110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.099949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn" event={"ID":"7e4dcb34-e6bd-48da-a121-dd77e4058a97","Type":"ContainerDied","Data":"e98245f1390776c4edd6ca26466c880e80c01bcdc1abded03039c217c47d8d47"}
Sep 11 18:30:04.100110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.100002    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e98245f1390776c4edd6ca26466c880e80c01bcdc1abded03039c217c47d8d47"
Sep 11 18:30:04.101590 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.101568    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"
Sep 11 18:30:04.101709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.101616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k" event={"ID":"89a0b8e7-bb2d-428a-876a-fbbefdf8ba06","Type":"ContainerDied","Data":"8f0feb7a5e7b14843cd93a7de1aa6fde0f753f5b40cbe9f9b46f94c3530606a0"}
Sep 11 18:30:04.101709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.101662    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f0feb7a5e7b14843cd93a7de1aa6fde0f753f5b40cbe9f9b46f94c3530606a0"
Sep 11 18:30:04.307600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:04.307567    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:05.035275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.035224    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:05.043428 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.043403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:30:05.207596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.207562    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:05.829555 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.829501    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:30:05.849312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849273    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-certs\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849326    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-data\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849383    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-podinfo\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849408    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-token\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-credential-socket\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istiod-ca-cert\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849550    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:30:05.849669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-data" (OuterVolumeSpecName: "istio-data") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:30:05.849669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849577    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-socket\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849644    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9skc\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-kube-api-access-x9skc\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-envoy\") pod \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\" (UID: \"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067\") "
Sep 11 18:30:05.849922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.849779    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:30:05.850148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.850102    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:30:05.850148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.850120    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.850148 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.850141    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.850408 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.850148    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:30:05.850408 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.850156    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.851693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.851664    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-token" (OuterVolumeSpecName: "istio-token") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:30:05.851809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.851782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-kube-api-access-x9skc" (OuterVolumeSpecName: "kube-api-access-x9skc") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "kube-api-access-x9skc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:30:05.852130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.852105    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:30:05.852214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.852196    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" (UID: "a98fe6a7-7c7b-45c8-b5c7-18e7a3057067"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:30:05.907321 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.907291    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:05.950666 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.950636    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.950799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.950738    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9skc\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-kube-api-access-x9skc\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.950799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.950754    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.950799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.950762    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.950799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.950770    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:05.950799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:05.950778    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:06.117328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.117209    2563 generic.go:356] "Generic (PLEG): container finished" podID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerID="0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463" exitCode=0
Sep 11 18:30:06.117328 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.117283    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"
Sep 11 18:30:06.117523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.117250    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" event={"ID":"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067","Type":"ContainerDied","Data":"0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463"}
Sep 11 18:30:06.117523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.117407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd" event={"ID":"a98fe6a7-7c7b-45c8-b5c7-18e7a3057067","Type":"ContainerDied","Data":"0b9e28d8dd41017255fbadbf074b6a03074b5781edb5e0a30cacf1185bb1b8cc"}
Sep 11 18:30:06.117523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.117434    2563 scope.go:117] "RemoveContainer" containerID="0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463"
Sep 11 18:30:06.127057 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.127036    2563 scope.go:117] "RemoveContainer" containerID="0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"
Sep 11 18:30:06.133175 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.133152    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"]
Sep 11 18:30:06.136509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.136492    2563 scope.go:117] "RemoveContainer" containerID="0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463"
Sep 11 18:30:06.136823 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.136802    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f63893-openshift-default-6d74695c9b-2hxfd"]
Sep 11 18:30:06.136879 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:30:06.136812    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463\": container with ID starting with 0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463 not found: ID does not exist" containerID="0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463"
Sep 11 18:30:06.136879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.136856    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463"} err="failed to get container status \"0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463\": rpc error: code = NotFound desc = could not find container \"0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463\": container with ID starting with 0451554e4f632a8ba23a05106a317842d459881cf6f8d092e3036971bc320463 not found: ID does not exist"
Sep 11 18:30:06.136879 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.136874    2563 scope.go:117] "RemoveContainer" containerID="0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"
Sep 11 18:30:06.137152 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:30:06.137135    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480\": container with ID starting with 0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480 not found: ID does not exist" containerID="0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"
Sep 11 18:30:06.137207 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.137155    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480"} err="failed to get container status \"0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480\": rpc error: code = NotFound desc = could not find container \"0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480\": container with ID starting with 0a2f829aaf16a5f9a33502603b317d2b67010ac26e1c5774c179a5ef2b246480 not found: ID does not exist"
Sep 11 18:30:06.628480 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:06.628447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:07.285795 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:07.285756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:07.465687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:07.465656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:07.914001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:07.913969    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" path="/var/lib/kubelet/pods/a98fe6a7-7c7b-45c8-b5c7-18e7a3057067/volumes"
Sep 11 18:30:08.134816 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:08.134778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:11.017243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.017209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:11.612213 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.612180    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:11.640943 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.640910    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"]
Sep 11 18:30:11.641235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.641207    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq" podUID="13fbfc94-e5a4-4a79-bd4e-12e757c81568" containerName="payload-processing" containerID="cri-o://eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9" gracePeriod=30
Sep 11 18:30:11.678184 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.678147    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:11.751486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.751457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:11.819422 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.818988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:11.903663 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:11.903635    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:30:12.005312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.005279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13fbfc94-e5a4-4a79-bd4e-12e757c81568-plugins-config-volume\") pod \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") "
Sep 11 18:30:12.005486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.005444    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmb2c\" (UniqueName: \"kubernetes.io/projected/13fbfc94-e5a4-4a79-bd4e-12e757c81568-kube-api-access-vmb2c\") pod \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\" (UID: \"13fbfc94-e5a4-4a79-bd4e-12e757c81568\") "
Sep 11 18:30:12.005714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.005685    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13fbfc94-e5a4-4a79-bd4e-12e757c81568-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13fbfc94-e5a4-4a79-bd4e-12e757c81568" (UID: "13fbfc94-e5a4-4a79-bd4e-12e757c81568"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:30:12.007609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.007575    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13fbfc94-e5a4-4a79-bd4e-12e757c81568-kube-api-access-vmb2c" (OuterVolumeSpecName: "kube-api-access-vmb2c") pod "13fbfc94-e5a4-4a79-bd4e-12e757c81568" (UID: "13fbfc94-e5a4-4a79-bd4e-12e757c81568"). InnerVolumeSpecName "kube-api-access-vmb2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:30:12.107094 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.107064    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13fbfc94-e5a4-4a79-bd4e-12e757c81568-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:12.107094 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.107088    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmb2c\" (UniqueName: \"kubernetes.io/projected/13fbfc94-e5a4-4a79-bd4e-12e757c81568-kube-api-access-vmb2c\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:30:12.143945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.143883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:12.158935 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.158904    2563 generic.go:356] "Generic (PLEG): container finished" podID="13fbfc94-e5a4-4a79-bd4e-12e757c81568" containerID="eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9" exitCode=0
Sep 11 18:30:12.159060 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.158952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq" event={"ID":"13fbfc94-e5a4-4a79-bd4e-12e757c81568","Type":"ContainerDied","Data":"eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9"}
Sep 11 18:30:12.159060 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.159009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq" event={"ID":"13fbfc94-e5a4-4a79-bd4e-12e757c81568","Type":"ContainerDied","Data":"fd2e00951381c543024f6ba13e65ee087db5922900376eed0066a294bccae6db"}
Sep 11 18:30:12.159060 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.159036    2563 scope.go:117] "RemoveContainer" containerID="eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9"
Sep 11 18:30:12.159060 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.158959    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"
Sep 11 18:30:12.170953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.170937    2563 scope.go:117] "RemoveContainer" containerID="eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9"
Sep 11 18:30:12.171167 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:30:12.171146    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9\": container with ID starting with eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9 not found: ID does not exist" containerID="eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9"
Sep 11 18:30:12.171311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.171171    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9"} err="failed to get container status \"eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9\": rpc error: code = NotFound desc = could not find container \"eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9\": container with ID starting with eb36bdaf136816e3d54cf8b50b6d30aad8f0b0e7dd78834fe70023c3941e65e9 not found: ID does not exist"
Sep 11 18:30:12.185810 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.185785    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"]
Sep 11 18:30:12.189484 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.189463    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-12eb53-86d9499f79-nh5pq"]
Sep 11 18:30:12.294575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:12.294541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:13.091753 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:13.091710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:13.810783 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:13.810748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:13.913021 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:13.912987    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13fbfc94-e5a4-4a79-bd4e-12e757c81568" path="/var/lib/kubelet/pods/13fbfc94-e5a4-4a79-bd4e-12e757c81568/volumes"
Sep 11 18:30:14.388142 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:14.388110    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:14.600014 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:14.599981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:15.583850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:15.583811    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:16.258113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.258080    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:16.355967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.355934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:16.588812 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.588780    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"]
Sep 11 18:30:16.589421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589405    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy"
Sep 11 18:30:16.589477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589424    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy"
Sep 11 18:30:16.589477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589437    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89a0b8e7-bb2d-428a-876a-fbbefdf8ba06" containerName="cleanup"
Sep 11 18:30:16.589477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589446    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89a0b8e7-bb2d-428a-876a-fbbefdf8ba06" containerName="cleanup"
Sep 11 18:30:16.589477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589467    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13fbfc94-e5a4-4a79-bd4e-12e757c81568" containerName="payload-processing"
Sep 11 18:30:16.589477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589475    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13fbfc94-e5a4-4a79-bd4e-12e757c81568" containerName="payload-processing"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589499    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589508    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589517    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4dcb34-e6bd-48da-a121-dd77e4058a97" containerName="cleanup"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589525    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4dcb34-e6bd-48da-a121-dd77e4058a97" containerName="cleanup"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589622    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589637    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a98fe6a7-7c7b-45c8-b5c7-18e7a3057067" containerName="istio-proxy"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589648    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4dcb34-e6bd-48da-a121-dd77e4058a97" containerName="cleanup"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589658    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="89a0b8e7-bb2d-428a-876a-fbbefdf8ba06" containerName="cleanup"
Sep 11 18:30:16.589665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.589668    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="13fbfc94-e5a4-4a79-bd4e-12e757c81568" containerName="payload-processing"
Sep 11 18:30:16.594539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.594518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.597185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.597164    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3e24a782-openshift-default-dockercfg-rzrsg\""
Sep 11 18:30:16.605082 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.605058    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"]
Sep 11 18:30:16.625390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.625368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:16.648404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30c483cc-8729-4915-8899-385bd6ef2e72-istiod-ca-cert\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648503 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-certs\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648503 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648456    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-istio-token\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-credential-socket\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-envoy\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr6zt\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-kube-api-access-hr6zt\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648750 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30c483cc-8729-4915-8899-385bd6ef2e72-istio-podinfo\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648750 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-socket\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.648850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.648812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-data\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.749855 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-data\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30c483cc-8729-4915-8899-385bd6ef2e72-istiod-ca-cert\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-certs\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-istio-token\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-credential-socket\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-envoy\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.749989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr6zt\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-kube-api-access-hr6zt\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30c483cc-8729-4915-8899-385bd6ef2e72-istio-podinfo\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-socket\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-certs\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750559 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-credential-socket\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750658    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-socket\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30c483cc-8729-4915-8899-385bd6ef2e72-istiod-ca-cert\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.750827 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.750728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-data\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.752858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.752834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-envoy\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.754051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.753927    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30c483cc-8729-4915-8899-385bd6ef2e72-istio-podinfo\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.758686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.758662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-istio-token\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.758810 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.758708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr6zt\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-kube-api-access-hr6zt\") pod \"e2e-mt-3e24a782-openshift-default-58df774844-lxqrn\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") " pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:16.783275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.783242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:16.907350 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:16.907287    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:17.184095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:17.184063    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"]
Sep 11 18:30:17.186161 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:30:17.186134    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c483cc_8729_4915_8899_385bd6ef2e72.slice/crio-dd597bdbc7f33ac84ea750ff00e63d555402e20fc6ac5a160845e7c64667e86d WatchSource:0}: Error finding container dd597bdbc7f33ac84ea750ff00e63d555402e20fc6ac5a160845e7c64667e86d: Status 404 returned error can't find the container with id dd597bdbc7f33ac84ea750ff00e63d555402e20fc6ac5a160845e7c64667e86d
Sep 11 18:30:17.188606 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:17.188562    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:17.188710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:17.188638    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:17.188710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:17.188680    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:18.193495 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.193466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" event={"ID":"30c483cc-8729-4915-8899-385bd6ef2e72","Type":"ContainerStarted","Data":"9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"}
Sep 11 18:30:18.193916 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.193512    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" event={"ID":"30c483cc-8729-4915-8899-385bd6ef2e72","Type":"ContainerStarted","Data":"dd597bdbc7f33ac84ea750ff00e63d555402e20fc6ac5a160845e7c64667e86d"}
Sep 11 18:30:18.214701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.214657    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podStartSLOduration=2.214642096 podStartE2EDuration="2.214642096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:18.213579527 +0000 UTC m=+1186.806780086" watchObservedRunningTime="2026-09-11 18:30:18.214642096 +0000 UTC m=+1186.807842653"
Sep 11 18:30:18.581227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.581197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:18.681075 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.681046    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:18.833585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.833504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:18.907500 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.907462    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:18.912582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:18.912558    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:19.200015 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.199930    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:19.200936 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.200904    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:19.351493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.351461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:19.470371 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.470286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:19.638361 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.638323    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"]
Sep 11 18:30:19.642383 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.642360    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.644761 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.644740    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-bf9898-openshift-default-dockercfg-s9vx8\""
Sep 11 18:30:19.653686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.653660    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"]
Sep 11 18:30:19.677527 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677501    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51468e39-25ed-4574-9768-3fe097f31a53-istio-podinfo\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.677661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-istio-token\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.677661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51468e39-25ed-4574-9768-3fe097f31a53-istiod-ca-cert\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.677661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-credential-socket\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.677848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-certs\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.677848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-data\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.677848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nppx5\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-kube-api-access-nppx5\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.678021 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-envoy\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.678021 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.677909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-socket\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.679477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.679453    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:19.778577 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-data\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nppx5\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-kube-api-access-nppx5\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-envoy\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-socket\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778742 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51468e39-25ed-4574-9768-3fe097f31a53-istio-podinfo\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-istio-token\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778787    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51468e39-25ed-4574-9768-3fe097f31a53-istiod-ca-cert\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.778967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-credential-socket\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.779117 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.778973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-data\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.779117 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.779012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-certs\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.779297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.779275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-credential-socket\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.779380 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.779318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-certs\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.779541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.779517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-socket\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.779701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.779672    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51468e39-25ed-4574-9768-3fe097f31a53-istiod-ca-cert\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.781356 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.781327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-envoy\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.781654 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.781635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51468e39-25ed-4574-9768-3fe097f31a53-istio-podinfo\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.789140 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.789115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nppx5\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-kube-api-access-nppx5\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.789622 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.789602    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-istio-token\") pod \"e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") " pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.956149 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.956123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:19.962357 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:19.962330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:20.105386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.105295    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:20.201167 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.201128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:20.238221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.238201    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"]
Sep 11 18:30:20.239946 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:30:20.239923    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51468e39_25ed_4574_9768_3fe097f31a53.slice/crio-54dcfadca859013dd8127619a621737b580a23d0bd2ab605111f3e366e7d0d42 WatchSource:0}: Error finding container 54dcfadca859013dd8127619a621737b580a23d0bd2ab605111f3e366e7d0d42: Status 404 returned error can't find the container with id 54dcfadca859013dd8127619a621737b580a23d0bd2ab605111f3e366e7d0d42
Sep 11 18:30:20.242108 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.242077    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:20.242183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.242139    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:20.242183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.242169    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:20.627145 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:20.627112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:21.212573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:21.212533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" event={"ID":"51468e39-25ed-4574-9768-3fe097f31a53","Type":"ContainerStarted","Data":"0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"}
Sep 11 18:30:21.212990 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:21.212594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" event={"ID":"51468e39-25ed-4574-9768-3fe097f31a53","Type":"ContainerStarted","Data":"54dcfadca859013dd8127619a621737b580a23d0bd2ab605111f3e366e7d0d42"}
Sep 11 18:30:21.238425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:21.238373    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podStartSLOduration=2.238358285 podStartE2EDuration="2.238358285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:21.236769295 +0000 UTC m=+1189.829969851" watchObservedRunningTime="2026-09-11 18:30:21.238358285 +0000 UTC m=+1189.831558841"
Sep 11 18:30:21.546871 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:21.546837    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:21.956913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:21.956842    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:21.961601 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:21.961577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:22.219352 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:22.219240    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:22.220218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:22.220187    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:22.496413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:22.496336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:23.133225 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:23.133192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:23.219507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:23.219473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:24.115926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:24.115891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:24.217185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:24.217142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:24.361568 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:24.361529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:24.821290 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:24.821236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:24.952947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:24.952919    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:25.319606 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.319571    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:25.417979 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.417945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:25.714679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.714601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:25.814873 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.814838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:25.915711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.915682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:25.942184 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.942157    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"]
Sep 11 18:30:25.946330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.946313    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:25.948718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.948697    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-bf9898\""
Sep 11 18:30:25.948828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.948699    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-bf9898-dockercfg-czg6k\""
Sep 11 18:30:25.957284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:25.957251    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"]
Sep 11 18:30:26.044888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.044841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.045297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.045247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpdg4\" (UniqueName: \"kubernetes.io/projected/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-kube-api-access-tpdg4\") pod \"payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.146509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.146476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.146670 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.146569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpdg4\" (UniqueName: \"kubernetes.io/projected/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-kube-api-access-tpdg4\") pod \"payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.147242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.147223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.155663 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.155641    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpdg4\" (UniqueName: \"kubernetes.io/projected/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-kube-api-access-tpdg4\") pod \"payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.258736 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.258704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:26.299786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.299680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:26.533315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:26.533290    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"]
Sep 11 18:30:26.535471 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:30:26.535441    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8833bc1b_04fb_4f06_9af4_6b6ff28e4c99.slice/crio-dafb4298fd86c4c03f49dcaf722db37c0d7274cfe64ce9766d0dfeb485d67260 WatchSource:0}: Error finding container dafb4298fd86c4c03f49dcaf722db37c0d7274cfe64ce9766d0dfeb485d67260: Status 404 returned error can't find the container with id dafb4298fd86c4c03f49dcaf722db37c0d7274cfe64ce9766d0dfeb485d67260
Sep 11 18:30:27.114713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:27.114681    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:27.255792 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:27.255748    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6" event={"ID":"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99","Type":"ContainerStarted","Data":"c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962"}
Sep 11 18:30:27.255953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:27.255854    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:27.255953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:27.255884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6" event={"ID":"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99","Type":"ContainerStarted","Data":"dafb4298fd86c4c03f49dcaf722db37c0d7274cfe64ce9766d0dfeb485d67260"}
Sep 11 18:30:27.283384 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:27.283338    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6" podStartSLOduration=2.283324534 podStartE2EDuration="2.283324534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:27.282028579 +0000 UTC m=+1195.875229166" watchObservedRunningTime="2026-09-11 18:30:27.283324534 +0000 UTC m=+1195.876525109"
Sep 11 18:30:27.781185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:27.781150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:28.612154 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:28.612128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:29.285292 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:29.285245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:30.085111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:30.085067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:31.940679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:31.940650    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:30:31.942772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:31.942748    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:30:32.591823 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:32.591787    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:38.263426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:38.263396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:30:45.526221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:45.526182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:46.424142 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:46.424077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:47.199498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:47.199465    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:47.942745 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:47.942706    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:48.078687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.078641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:48.400394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.400365    2563 generic.go:356] "Generic (PLEG): container finished" podID="51468e39-25ed-4574-9768-3fe097f31a53" containerID="0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821" exitCode=0
Sep 11 18:30:48.400875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.400402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" event={"ID":"51468e39-25ed-4574-9768-3fe097f31a53","Type":"ContainerDied","Data":"0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"}
Sep 11 18:30:48.400875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.400786    2563 scope.go:117] "RemoveContainer" containerID="0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"
Sep 11 18:30:48.402323 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.402249    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:48.402412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.402387    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:48.402460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.402441    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:48.412415 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.412395    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:48.709558 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.709473    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"]
Sep 11 18:30:48.713728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.713706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.716730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.716704    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-5faf38-openshift-default-dockercfg-86hvk\""
Sep 11 18:30:48.721521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.721482    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:48.728463 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.728439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"]
Sep 11 18:30:48.776792 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.776756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-data\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.776938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.776850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-token\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.776982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.776939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-podinfo\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.776982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.776972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istiod-ca-cert\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.777084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.777025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-envoy\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.777084 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.777075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9c84\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-kube-api-access-z9c84\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.777187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.777127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-credential-socket\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.777338 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.777319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-certs\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.777392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.777358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-socket\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.816748 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.816593    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:48.878349 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-envoy\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878363    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9c84\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-kube-api-access-z9c84\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-credential-socket\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-certs\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-socket\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-data\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-token\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-podinfo\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istiod-ca-cert\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-certs\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.878976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.878962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-socket\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.879204 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.879156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-credential-socket\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.879310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.879211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-data\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.879451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.879431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istiod-ca-cert\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.881449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.881416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-podinfo\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.881602 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.881505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-envoy\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.887131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.887111    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9c84\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-kube-api-access-z9c84\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:48.887988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:48.887964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-token\") pod \"e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") " pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:49.028966 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.028934    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:49.032802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.032776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:49.223905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.223808    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 11 18:30:49.223905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.223884    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 11 18:30:49.318655 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.316186    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"]
Sep 11 18:30:49.325607 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.325544    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:49.325747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.325630    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:49.325747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.325679    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:49.422287 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.421511    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" event={"ID":"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66","Type":"ContainerStarted","Data":"58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"}
Sep 11 18:30:49.422287 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.421582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" event={"ID":"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66","Type":"ContainerStarted","Data":"113386a8ddbb6baa5143fc8c8c3cd1d27c364127fb26572b3b423346f7b039cb"}
Sep 11 18:30:49.429255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.429212    2563 generic.go:356] "Generic (PLEG): container finished" podID="30c483cc-8729-4915-8899-385bd6ef2e72" containerID="9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e" exitCode=0
Sep 11 18:30:49.429457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.429304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" event={"ID":"30c483cc-8729-4915-8899-385bd6ef2e72","Type":"ContainerDied","Data":"9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"}
Sep 11 18:30:49.429969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.429790    2563 scope.go:117] "RemoveContainer" containerID="9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"
Sep 11 18:30:49.431629 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.431593    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:49.431752 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.431701    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:49.431752 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.431743    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:30:49.433290 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.432732    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" event={"ID":"51468e39-25ed-4574-9768-3fe097f31a53","Type":"ContainerStarted","Data":"ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00"}
Sep 11 18:30:49.451331 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.451255    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podStartSLOduration=1.451234281 podStartE2EDuration="1.451234281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:30:49.44569873 +0000 UTC m=+1218.038899303" watchObservedRunningTime="2026-09-11 18:30:49.451234281 +0000 UTC m=+1218.044434839"
Sep 11 18:30:49.845331 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.845295    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:49.956553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.956519    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:49.957599 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.957573    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 18:30:49.957723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:49.957621    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 18:30:50.029486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.029456    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:50.034648 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.034622    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:50.199626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.199533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:50.444985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.444953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" event={"ID":"30c483cc-8729-4915-8899-385bd6ef2e72","Type":"ContainerStarted","Data":"37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c"}
Sep 11 18:30:50.447246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.447223    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:50.448608 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.448582    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:30:50.908068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.908026    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:50.908311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.908195    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 11 18:30:50.908311 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.908245    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 11 18:30:50.957674 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.957632    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 18:30:50.957840 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:50.957723    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 18:30:51.052447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.052400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:51.429226 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.429156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:51.594013 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.593982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:51.907834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.907799    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 11 18:30:51.908104 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.908074    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 11 18:30:51.956665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.956627    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 18:30:51.956829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:51.956683    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 18:30:52.203806 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:52.203711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:52.219759 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:52.219728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:52.907887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:52.907843    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 11 18:30:52.908380 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:52.907911    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 11 18:30:52.957320 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:52.957243    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 18:30:52.957497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:52.957343    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 18:30:53.240005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:53.239905    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:53.912792 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:53.912752    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 11 18:30:53.913296 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:53.912813    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 11 18:30:53.957129 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:53.957097    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 18:30:53.957295 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:53.957145    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 18:30:54.113477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:54.113440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:54.926284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:54.924842    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 11 18:30:54.926284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:54.924901    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 11 18:30:54.961675 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:54.961647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:54.962638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:54.962618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:30:54.985488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:54.985456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:55.082620 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.082591    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:55.530029 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.529988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:55.625466 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.625430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:55.914817 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.914747    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:55.915176 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.915151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:55.915809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.915788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:30:55.940948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:55.940923    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:56.010080 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:56.010038    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:56.801557 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:56.801526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:57.527468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:57.527425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:30:58.197731 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:30:58.197693    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:06.288941 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.288913    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"]
Sep 11 18:31:06.292448 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.292430    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.294618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.294597    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d73031\"/\"distinct-2-w0-d73031-kserve-self-signed-certs\""
Sep 11 18:31:06.303204 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.303179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"]
Sep 11 18:31:06.311945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.311921    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:06.374083 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.374052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tmp-dir\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.374197 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.374119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-dshm\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.374197 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.374149    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvcf9\" (UniqueName: \"kubernetes.io/projected/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-kube-api-access-hvcf9\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.374358 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.374196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-home\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.374358 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.374255    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tls-certs\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.374358 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.374335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-model-cache\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.474998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.474972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tmp-dir\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475105 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-dshm\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475105 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvcf9\" (UniqueName: \"kubernetes.io/projected/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-kube-api-access-hvcf9\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475105 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-home\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475119    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tls-certs\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-model-cache\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475464 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tmp-dir\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475540 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475517    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-home\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.475577 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.475539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-model-cache\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.477242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.477226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-dshm\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.477594 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.477575    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tls-certs\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.482513 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.482489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvcf9\" (UniqueName: \"kubernetes.io/projected/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-kube-api-access-hvcf9\") pod \"distinct-2-w0-d73031-kserve-866984799d-6hlmc\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") " pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.602706 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.602632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:06.878297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:06.878241    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"]
Sep 11 18:31:06.880028 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:31:06.880003    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7f7e4ff_eefa_4e72_9aa1_6057861c9a55.slice/crio-92a3773c8e431a5a646031ab8cee0f9e8cd27a06cac93298b91f4ba71161767e WatchSource:0}: Error finding container 92a3773c8e431a5a646031ab8cee0f9e8cd27a06cac93298b91f4ba71161767e: Status 404 returned error can't find the container with id 92a3773c8e431a5a646031ab8cee0f9e8cd27a06cac93298b91f4ba71161767e
Sep 11 18:31:07.541134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:07.541104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" event={"ID":"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55","Type":"ContainerStarted","Data":"70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8"}
Sep 11 18:31:07.541493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:07.541144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" event={"ID":"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55","Type":"ContainerStarted","Data":"92a3773c8e431a5a646031ab8cee0f9e8cd27a06cac93298b91f4ba71161767e"}
Sep 11 18:31:07.559097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:07.559025    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" podStartSLOduration=1.558991762 podStartE2EDuration="1.558991762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:31:07.557964814 +0000 UTC m=+1236.151165372" watchObservedRunningTime="2026-09-11 18:31:07.558991762 +0000 UTC m=+1236.152192320"
Sep 11 18:31:08.687996 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:08.687964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:16.603216 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:16.603179    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:16.603741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:16.603240    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:16.616371 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:16.616343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:17.098739 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:17.098700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:17.614159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:17.614132    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:31:17.899233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:17.899152    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:18.583605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:18.583573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:19.289587 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.289557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:19.610457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.610428    2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerID="58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35" exitCode=0
Sep 11 18:31:19.610680 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.610512    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" event={"ID":"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66","Type":"ContainerDied","Data":"58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"}
Sep 11 18:31:19.611010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.610993    2563 scope.go:117] "RemoveContainer" containerID="58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"
Sep 11 18:31:19.612388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.612354    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:31:19.612498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.612439    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:31:19.612498 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:19.612490    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:31:20.217539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:20.217502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:20.447476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:20.447423    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:31:20.625924 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:20.625879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" event={"ID":"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66","Type":"ContainerStarted","Data":"1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6"}
Sep 11 18:31:21.029183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:21.029147    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:31:21.030231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:21.030199    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 18:31:21.030386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:21.030251    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 18:31:21.113918 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:21.113881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:21.474878 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:21.474801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:22.029987 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:22.029955    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 18:31:22.030160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:22.030015    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 18:31:22.095681 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:22.095651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:22.867276 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:22.867231    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:23.029249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:23.029216    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 18:31:23.029448 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:23.029296    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 18:31:24.029404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:24.029320    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 18:31:24.029404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:24.029379    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 18:31:25.029732 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:25.029695    2563 patch_prober.go:28] interesting pod/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Sep 11 18:31:25.030097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:25.029754    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Sep 11 18:31:26.033553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:26.033517    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:31:26.033981 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:26.033799    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:31:26.034411 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:26.034388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:31:32.023354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:32.023325    2563 scope.go:117] "RemoveContainer" containerID="0ffcd0b1578e62c09b3a986b6942c2995c0f1c1ec7a34f833026bde733de38aa"
Sep 11 18:31:34.550681 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:34.550639    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:59.712662 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:59.712631    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:31:59.776736 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:31:59.776711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:13.967210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:13.967174    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:14.425674 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:14.425636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:18.048324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:18.048284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:23.385988 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:23.385955    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"]
Sep 11 18:32:23.408109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:23.408081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:38.405620 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.405577    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs" podUID="0a70ce38-609c-4eae-b19a-08a17d738824" containerName="main" containerID="cri-o://3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc" gracePeriod=45
Sep 11 18:32:38.645294 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.645240    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:32:38.747039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.746975    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-tmp-dir\") pod \"0a70ce38-609c-4eae-b19a-08a17d738824\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") "
Sep 11 18:32:38.747039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-dshm\") pod \"0a70ce38-609c-4eae-b19a-08a17d738824\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") "
Sep 11 18:32:38.747205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747162    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a70ce38-609c-4eae-b19a-08a17d738824-tls-certs\") pod \"0a70ce38-609c-4eae-b19a-08a17d738824\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") "
Sep 11 18:32:38.747242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-model-cache\") pod \"0a70ce38-609c-4eae-b19a-08a17d738824\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") "
Sep 11 18:32:38.747308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747248    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a70ce38-609c-4eae-b19a-08a17d738824" (UID: "0a70ce38-609c-4eae-b19a-08a17d738824"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:38.747308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747294    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-home\") pod \"0a70ce38-609c-4eae-b19a-08a17d738824\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") "
Sep 11 18:32:38.747376 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747348    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c52qh\" (UniqueName: \"kubernetes.io/projected/0a70ce38-609c-4eae-b19a-08a17d738824-kube-api-access-c52qh\") pod \"0a70ce38-609c-4eae-b19a-08a17d738824\" (UID: \"0a70ce38-609c-4eae-b19a-08a17d738824\") "
Sep 11 18:32:38.747530 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747511    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a70ce38-609c-4eae-b19a-08a17d738824" (UID: "0a70ce38-609c-4eae-b19a-08a17d738824"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:38.747598 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747560    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-home" (OuterVolumeSpecName: "home") pod "0a70ce38-609c-4eae-b19a-08a17d738824" (UID: "0a70ce38-609c-4eae-b19a-08a17d738824"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:38.747751 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747735    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:32:38.747809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747753    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:32:38.747809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.747765    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:32:38.749344 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.749319    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-dshm" (OuterVolumeSpecName: "dshm") pod "0a70ce38-609c-4eae-b19a-08a17d738824" (UID: "0a70ce38-609c-4eae-b19a-08a17d738824"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:32:38.749461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.749424    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a70ce38-609c-4eae-b19a-08a17d738824-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a70ce38-609c-4eae-b19a-08a17d738824" (UID: "0a70ce38-609c-4eae-b19a-08a17d738824"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:32:38.749461 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.749430    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a70ce38-609c-4eae-b19a-08a17d738824-kube-api-access-c52qh" (OuterVolumeSpecName: "kube-api-access-c52qh") pod "0a70ce38-609c-4eae-b19a-08a17d738824" (UID: "0a70ce38-609c-4eae-b19a-08a17d738824"). InnerVolumeSpecName "kube-api-access-c52qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:32:38.849032 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.849010    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a70ce38-609c-4eae-b19a-08a17d738824-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:32:38.849032 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.849033    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c52qh\" (UniqueName: \"kubernetes.io/projected/0a70ce38-609c-4eae-b19a-08a17d738824-kube-api-access-c52qh\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:32:38.849167 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:38.849043    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a70ce38-609c-4eae-b19a-08a17d738824-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:32:39.089754 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.089722    2563 generic.go:356] "Generic (PLEG): container finished" podID="0a70ce38-609c-4eae-b19a-08a17d738824" containerID="3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc" exitCode=0
Sep 11 18:32:39.089874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.089762    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"
Sep 11 18:32:39.089874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.089794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs" event={"ID":"0a70ce38-609c-4eae-b19a-08a17d738824","Type":"ContainerDied","Data":"3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc"}
Sep 11 18:32:39.089874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.089823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs" event={"ID":"0a70ce38-609c-4eae-b19a-08a17d738824","Type":"ContainerDied","Data":"2c0fe1fdaf26d60736b49c52ed1d0079a93b664fbd99f10f3ab2daced03c677a"}
Sep 11 18:32:39.089874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.089841    2563 scope.go:117] "RemoveContainer" containerID="3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc"
Sep 11 18:32:39.103206 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.103189    2563 scope.go:117] "RemoveContainer" containerID="3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc"
Sep 11 18:32:39.103506 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:32:39.103489    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc\": container with ID starting with 3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc not found: ID does not exist" containerID="3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc"
Sep 11 18:32:39.103575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.103516    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc"} err="failed to get container status \"3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc\": rpc error: code = NotFound desc = could not find container \"3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc\": container with ID starting with 3b231187b0227ae198fa540273b23a463f2d240ec68d4a7006004adc28cd1adc not found: ID does not exist"
Sep 11 18:32:39.112707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.112681    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"]
Sep 11 18:32:39.116571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.116550    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-553596/e2e-conflict-a-c17caa91-kserve-647bbdcc95-7w8cs"]
Sep 11 18:32:39.914518 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:39.914484    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a70ce38-609c-4eae-b19a-08a17d738824" path="/var/lib/kubelet/pods/0a70ce38-609c-4eae-b19a-08a17d738824/volumes"
Sep 11 18:32:44.502796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.502762    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"]
Sep 11 18:32:44.503235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.503222    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a70ce38-609c-4eae-b19a-08a17d738824" containerName="main"
Sep 11 18:32:44.503303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.503236    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a70ce38-609c-4eae-b19a-08a17d738824" containerName="main"
Sep 11 18:32:44.503346 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.503320    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a70ce38-609c-4eae-b19a-08a17d738824" containerName="main"
Sep 11 18:32:44.506233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.506213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.509354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.509324    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"openshift-service-ca.crt\""
Sep 11 18:32:44.509354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.509348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"default-dockercfg-j5lh6\""
Sep 11 18:32:44.509354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.509326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"distinct-w4-553596-kserve-self-signed-certs\""
Sep 11 18:32:44.509546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.509348    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-553596\"/\"kube-root-ca.crt\""
Sep 11 18:32:44.515418 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.515384    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"]
Sep 11 18:32:44.527959 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.527929    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:44.602010 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.601980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-model-cache\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.602125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.602059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-home\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.602125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.602080    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-dshm\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.602125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.602099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tls-certs\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.602243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.602170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tmp-dir\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.602243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.602215    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ft5m\" (UniqueName: \"kubernetes.io/projected/21f1a971-6336-4a0b-8cac-2f10b64fec3c-kube-api-access-5ft5m\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.703788 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.703754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ft5m\" (UniqueName: \"kubernetes.io/projected/21f1a971-6336-4a0b-8cac-2f10b64fec3c-kube-api-access-5ft5m\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.703956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.703814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-model-cache\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.703956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.703936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-home\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.704106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.703964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-dshm\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.704106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.703998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tls-certs\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.704106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.704074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tmp-dir\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.704347 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.704323    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-model-cache\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.704774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.704753    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-home\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.705130 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.705104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tmp-dir\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.707238 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.706974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-dshm\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.707367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.707346    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tls-certs\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.714541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.714515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ft5m\" (UniqueName: \"kubernetes.io/projected/21f1a971-6336-4a0b-8cac-2f10b64fec3c-kube-api-access-5ft5m\") pod \"distinct-w4-553596-kserve-76996b7866-22khv\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") " pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:44.816574 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:44.816539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:45.097005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:45.096979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"]
Sep 11 18:32:45.098328 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:32:45.098305    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21f1a971_6336_4a0b_8cac_2f10b64fec3c.slice/crio-4409a06bfd155da95ba639c45d1c50693f6a229b898708bde53d33668a39b113 WatchSource:0}: Error finding container 4409a06bfd155da95ba639c45d1c50693f6a229b898708bde53d33668a39b113: Status 404 returned error can't find the container with id 4409a06bfd155da95ba639c45d1c50693f6a229b898708bde53d33668a39b113
Sep 11 18:32:45.125109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:45.125084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv" event={"ID":"21f1a971-6336-4a0b-8cac-2f10b64fec3c","Type":"ContainerStarted","Data":"4409a06bfd155da95ba639c45d1c50693f6a229b898708bde53d33668a39b113"}
Sep 11 18:32:46.132663 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:46.132628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv" event={"ID":"21f1a971-6336-4a0b-8cac-2f10b64fec3c","Type":"ContainerStarted","Data":"d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa"}
Sep 11 18:32:46.154686 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:46.154639    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv" podStartSLOduration=2.154624397 podStartE2EDuration="2.154624397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:32:46.152138375 +0000 UTC m=+1334.745338946" watchObservedRunningTime="2026-09-11 18:32:46.154624397 +0000 UTC m=+1334.747825004"
Sep 11 18:32:52.441781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:52.441747    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:53.272650 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:53.272615    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:54.018631 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:54.018555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:54.790205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:54.790172    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:54.817336 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:54.817307    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:54.817469 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:54.817352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:54.829785 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:54.829765    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:55.193094 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:55.193010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:32:55.489432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:55.489355    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:56.254897 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:56.254865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:56.982244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:56.982208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:57.079559 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:57.079529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:57.669247 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:57.669215    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:32:58.497044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:32:58.497010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:07.511781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:07.511746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:07.590608 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:07.590579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:07.650983 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:07.650951    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:07.737678 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:07.737650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:07.808051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:07.808023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:08.206946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:08.206876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:08.922669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:08.922640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:09.609947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:09.609912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:10.402647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:10.402612    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:10.774453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:10.774375    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:11.097459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:11.097427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:11.784668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:11.784636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:12.597826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:12.597792    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:15.845719 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:15.845686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:41.802128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:41.802090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:51.727065 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:51.727027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:53.247834 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:53.247803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:53.775452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:53.775407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:56.286952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:56.286909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:33:59.693460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:33:59.693429    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:05.854359 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:05.854320    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:05.954636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:05.954608    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:06.147489 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:06.147412    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"]
Sep 11 18:34:06.147623 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:06.147600    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" containerID="cri-o://37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c" gracePeriod=30
Sep 11 18:34:06.184151 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:06.184120    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:07.359332 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:07.359299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:07.455188 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:07.455160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:07.531561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:07.531526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:07.605592 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:07.605555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:08.173118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:08.173087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:08.273865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:08.273839    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:08.369243 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:08.369213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:09.623563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:09.623536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:10.917661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:10.917627    2563 patch_prober.go:28] interesting pod/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:34:10.918031 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:10.917684    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:34:11.404312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.404289    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:34:11.417218 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417195    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-istio-token\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417326 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-data\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417326 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-credential-socket\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417326 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417290    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hr6zt\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-kube-api-access-hr6zt\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-envoy\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30c483cc-8729-4915-8899-385bd6ef2e72-istiod-ca-cert\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-certs\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417415    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30c483cc-8729-4915-8899-385bd6ef2e72-istio-podinfo\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417460    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-socket\") pod \"30c483cc-8729-4915-8899-385bd6ef2e72\" (UID: \"30c483cc-8729-4915-8899-385bd6ef2e72\") "
Sep 11 18:34:11.417711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417503    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-data" (OuterVolumeSpecName: "istio-data") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:11.417711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417522    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:11.417818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:11.417818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417770    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:11.417818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.417792    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30c483cc-8729-4915-8899-385bd6ef2e72-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:34:11.418037 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.418020    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.418103 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.418043    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.418103 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.418055    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30c483cc-8729-4915-8899-385bd6ef2e72-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.418103 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.418066    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.418103 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.418079    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.419447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.419416    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-istio-token" (OuterVolumeSpecName: "istio-token") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:11.419649 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.419626    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30c483cc-8729-4915-8899-385bd6ef2e72-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:34:11.419938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.419911    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-kube-api-access-hr6zt" (OuterVolumeSpecName: "kube-api-access-hr6zt") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "kube-api-access-hr6zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:34:11.419938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.419908    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30c483cc-8729-4915-8899-385bd6ef2e72" (UID: "30c483cc-8729-4915-8899-385bd6ef2e72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:34:11.519657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.519601    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30c483cc-8729-4915-8899-385bd6ef2e72-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.519657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.519627    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.519657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.519636    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hr6zt\" (UniqueName: \"kubernetes.io/projected/30c483cc-8729-4915-8899-385bd6ef2e72-kube-api-access-hr6zt\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.519657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.519644    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30c483cc-8729-4915-8899-385bd6ef2e72-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:34:11.612515 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.612486    2563 generic.go:356] "Generic (PLEG): container finished" podID="30c483cc-8729-4915-8899-385bd6ef2e72" containerID="37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c" exitCode=0
Sep 11 18:34:11.612626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.612529    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"
Sep 11 18:34:11.612626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.612562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" event={"ID":"30c483cc-8729-4915-8899-385bd6ef2e72","Type":"ContainerDied","Data":"37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c"}
Sep 11 18:34:11.612626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.612623    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn" event={"ID":"30c483cc-8729-4915-8899-385bd6ef2e72","Type":"ContainerDied","Data":"dd597bdbc7f33ac84ea750ff00e63d555402e20fc6ac5a160845e7c64667e86d"}
Sep 11 18:34:11.612737 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.612651    2563 scope.go:117] "RemoveContainer" containerID="37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c"
Sep 11 18:34:11.622202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.622185    2563 scope.go:117] "RemoveContainer" containerID="9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"
Sep 11 18:34:11.630168 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.630153    2563 scope.go:117] "RemoveContainer" containerID="37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c"
Sep 11 18:34:11.630461 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:34:11.630443    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c\": container with ID starting with 37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c not found: ID does not exist" containerID="37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c"
Sep 11 18:34:11.630510 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.630469    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c"} err="failed to get container status \"37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c\": rpc error: code = NotFound desc = could not find container \"37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c\": container with ID starting with 37aed1b4908c3a554001ef908d100578da7958f60c4f67ed3454451206f91c2c not found: ID does not exist"
Sep 11 18:34:11.630510 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.630488    2563 scope.go:117] "RemoveContainer" containerID="9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"
Sep 11 18:34:11.630724 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:34:11.630710    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e\": container with ID starting with 9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e not found: ID does not exist" containerID="9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"
Sep 11 18:34:11.630762 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.630728    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e"} err="failed to get container status \"9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e\": rpc error: code = NotFound desc = could not find container \"9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e\": container with ID starting with 9f16da8fa1831d36dfea81c2df071221a615ea56160374f6feb1eed9ffba042e not found: ID does not exist"
Sep 11 18:34:11.635789 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.635770    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"]
Sep 11 18:34:11.639698 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.639675    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3e24a782-openshift-default-58df774844-lxqrn"]
Sep 11 18:34:11.913323 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:11.913295    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" path="/var/lib/kubelet/pods/30c483cc-8729-4915-8899-385bd6ef2e72/volumes"
Sep 11 18:34:16.050183 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:16.050149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:16.120446 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:16.120421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:16.692436 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:16.692403    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:23.262160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:23.262129    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:23.365071 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:23.365044    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:23.462477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:23.462445    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:33.927435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:33.927407    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:34.142314 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:34.142283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:41.785244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:41.785204    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:34:55.039056 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:34:55.039022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:01.288101 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.288068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:01.354061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.354030    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:01.692031 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.691954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:01.928316 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928278    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"]
Sep 11 18:35:01.928738 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928722    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy"
Sep 11 18:35:01.928738 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928738    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy"
Sep 11 18:35:01.928858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928754    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy"
Sep 11 18:35:01.928858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928759    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy"
Sep 11 18:35:01.928858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928836    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy"
Sep 11 18:35:01.928858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.928844    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c483cc-8729-4915-8899-385bd6ef2e72" containerName="istio-proxy"
Sep 11 18:35:01.931949 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.931932    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.934435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.934410    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1e370a-openshift-default-dockercfg-gmc7k\""
Sep 11 18:35:01.940766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.940711    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"]
Sep 11 18:35:01.945664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.945589    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:01.987413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-certs\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987564 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-podinfo\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987564 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-envoy\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987579    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-socket\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-data\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987772 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-762xt\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-kube-api-access-762xt\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-credential-socket\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istiod-ca-cert\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:01.987907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:01.987886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-token\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.088802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.088763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-762xt\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-kube-api-access-762xt\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.088985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.088815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-credential-socket\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.088985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.088841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istiod-ca-cert\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.088985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.088869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-token\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.088985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.088904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-certs\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.088988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-podinfo\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-envoy\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-socket\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089210 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-data\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089522 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-certs\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089522 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089501    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-credential-socket\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-socket\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istiod-ca-cert\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.089828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.089807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-data\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.091384 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.091363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-envoy\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.091631 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.091612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-podinfo\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.101039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.101016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-token\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.101289 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.101244    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-762xt\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-kube-api-access-762xt\") pod \"e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") " pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.186160 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.186131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:02.245227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.245143    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:02.526850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.526823    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"]
Sep 11 18:35:02.528095 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:35:02.528071    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2d2943_36ae_43df_b7e4_bf5fb239049c.slice/crio-182f41733ae53efbe8f5ab40c50c8943689ad671b2321f4aec6afebb1d4fc7b4 WatchSource:0}: Error finding container 182f41733ae53efbe8f5ab40c50c8943689ad671b2321f4aec6afebb1d4fc7b4: Status 404 returned error can't find the container with id 182f41733ae53efbe8f5ab40c50c8943689ad671b2321f4aec6afebb1d4fc7b4
Sep 11 18:35:02.530582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.530543    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:02.530643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.530625    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:02.530685 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.530668    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:02.894110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.894075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" event={"ID":"4c2d2943-36ae-43df-b7e4-bf5fb239049c","Type":"ContainerStarted","Data":"cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"}
Sep 11 18:35:02.894295 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.894136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" event={"ID":"4c2d2943-36ae-43df-b7e4-bf5fb239049c","Type":"ContainerStarted","Data":"182f41733ae53efbe8f5ab40c50c8943689ad671b2321f4aec6afebb1d4fc7b4"}
Sep 11 18:35:02.915434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:02.915388    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podStartSLOduration=1.915370814 podStartE2EDuration="1.915370814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:02.912463508 +0000 UTC m=+1471.505664066" watchObservedRunningTime="2026-09-11 18:35:02.915370814 +0000 UTC m=+1471.508571371"
Sep 11 18:35:03.245483 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:03.245395    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:03.250815 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:03.250790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:03.900072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:03.900042    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:03.901005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:03.900985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:04.291884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:04.291845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:04.917220 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:04.917192    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:05.105248 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:05.105213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:05.258420 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:05.258343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:06.112764 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:06.112734    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:06.950427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:06.950397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:07.428009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:07.427971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:08.096200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:08.096171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:08.787854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:08.787820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:09.584964 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:09.584932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:10.396193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.396163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:10.496976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.496945    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:10.648083 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.648014    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:10.886120 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.886086    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:10.985154 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.985085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:10.986747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.986727    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"]
Sep 11 18:35:10.990122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.990103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:10.992587 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.992566    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1e370a\""
Sep 11 18:35:10.992707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.992588    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1e370a-dockercfg-grqnn\""
Sep 11 18:35:10.999900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:10.999880    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"]
Sep 11 18:35:11.084369 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.084334    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.084571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.084450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8m8k\" (UniqueName: \"kubernetes.io/projected/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-kube-api-access-d8m8k\") pod \"payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.185668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.185630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.185860 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.185683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8m8k\" (UniqueName: \"kubernetes.io/projected/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-kube-api-access-d8m8k\") pod \"payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.186348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.186325    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.198170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.198147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8m8k\" (UniqueName: \"kubernetes.io/projected/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-kube-api-access-d8m8k\") pod \"payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.302560 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.302530    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.389132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.389096    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:11.575548 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.575525    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"]
Sep 11 18:35:11.576921 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:35:11.576896    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e8bd2f_899c_4378_8172_4d5f6a0e3634.slice/crio-3a3ca4dade46788a4b51d4b1545191e44be600e8f8e65b5a05cc65f45b3d5150 WatchSource:0}: Error finding container 3a3ca4dade46788a4b51d4b1545191e44be600e8f8e65b5a05cc65f45b3d5150: Status 404 returned error can't find the container with id 3a3ca4dade46788a4b51d4b1545191e44be600e8f8e65b5a05cc65f45b3d5150
Sep 11 18:35:11.950541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.950465    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb" event={"ID":"d9e8bd2f-899c-4378-8172-4d5f6a0e3634","Type":"ContainerStarted","Data":"7fc2f5b5d3d2582eebb86c83ff43df4c05eefe9b15ebea0f359c61ad1dba7ceb"}
Sep 11 18:35:11.950541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.950509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb" event={"ID":"d9e8bd2f-899c-4378-8172-4d5f6a0e3634","Type":"ContainerStarted","Data":"3a3ca4dade46788a4b51d4b1545191e44be600e8f8e65b5a05cc65f45b3d5150"}
Sep 11 18:35:11.950541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.950534    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:11.967604 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:11.967557    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb" podStartSLOduration=1.96754353 podStartE2EDuration="1.96754353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:11.965920237 +0000 UTC m=+1480.559120798" watchObservedRunningTime="2026-09-11 18:35:11.96754353 +0000 UTC m=+1480.560744087"
Sep 11 18:35:12.295273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:12.295213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:12.401246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:12.401213    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"]
Sep 11 18:35:12.426125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:12.426091    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"]
Sep 11 18:35:12.430681 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:12.430657    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"]
Sep 11 18:35:12.538864 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:12.538834    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:13.193395 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:13.193362    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:14.020971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:14.020935    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:14.794354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:14.794326    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:15.499112 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:15.499078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:16.375746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:16.375716    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:17.706120 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:17.706089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:22.960438 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:22.960406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:35:23.302359 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.302325    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"]
Sep 11 18:35:23.302632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.302603    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" podUID="15df66de-fd57-4bbd-873d-6e759ec3f367" containerName="maas-api" containerID="cri-o://e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae" gracePeriod=30
Sep 11 18:35:23.334087 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.334051    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"]
Sep 11 18:35:23.337632 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.337604    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-d73031-29819190-ftg8k"]
Sep 11 18:35:23.359884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.359858    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"]
Sep 11 18:35:23.360140 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.360086    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh" podUID="0756087c-fe35-4c07-b3de-961ead072e49" containerName="payload-processing" containerID="cri-o://97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857" gracePeriod=30
Sep 11 18:35:23.367923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.367746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:23.392169 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:23.392139    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0756087c_fe35_4c07_b3de_961ead072e49.slice/crio-97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 18:35:23.430237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.430208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:23.531542 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.531182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:23.632572 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.632523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:23.641910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.641878    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:35:23.770668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.770644    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:35:23.809600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.809568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sshw\" (UniqueName: \"kubernetes.io/projected/0756087c-fe35-4c07-b3de-961ead072e49-kube-api-access-2sshw\") pod \"0756087c-fe35-4c07-b3de-961ead072e49\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") "
Sep 11 18:35:23.809777 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.809640    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0756087c-fe35-4c07-b3de-961ead072e49-plugins-config-volume\") pod \"0756087c-fe35-4c07-b3de-961ead072e49\" (UID: \"0756087c-fe35-4c07-b3de-961ead072e49\") "
Sep 11 18:35:23.810080 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.810046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0756087c-fe35-4c07-b3de-961ead072e49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0756087c-fe35-4c07-b3de-961ead072e49" (UID: "0756087c-fe35-4c07-b3de-961ead072e49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:35:23.810228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.810208    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0756087c-fe35-4c07-b3de-961ead072e49-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:23.811843 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.811819    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0756087c-fe35-4c07-b3de-961ead072e49-kube-api-access-2sshw" (OuterVolumeSpecName: "kube-api-access-2sshw") pod "0756087c-fe35-4c07-b3de-961ead072e49" (UID: "0756087c-fe35-4c07-b3de-961ead072e49"). InnerVolumeSpecName "kube-api-access-2sshw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:35:23.881119 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.881052    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:23.910590 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.910558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls\") pod \"15df66de-fd57-4bbd-873d-6e759ec3f367\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") "
Sep 11 18:35:23.910723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.910641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15df66de-fd57-4bbd-873d-6e759ec3f367-openshift-service-ca\") pod \"15df66de-fd57-4bbd-873d-6e759ec3f367\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") "
Sep 11 18:35:23.910723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.910679    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-metrics-certs\") pod \"15df66de-fd57-4bbd-873d-6e759ec3f367\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") "
Sep 11 18:35:23.910852 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.910801    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8jps\" (UniqueName: \"kubernetes.io/projected/15df66de-fd57-4bbd-873d-6e759ec3f367-kube-api-access-h8jps\") pod \"15df66de-fd57-4bbd-873d-6e759ec3f367\" (UID: \"15df66de-fd57-4bbd-873d-6e759ec3f367\") "
Sep 11 18:35:23.911061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.911034    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15df66de-fd57-4bbd-873d-6e759ec3f367-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15df66de-fd57-4bbd-873d-6e759ec3f367" (UID: "15df66de-fd57-4bbd-873d-6e759ec3f367"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:35:23.911334 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.911314    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sshw\" (UniqueName: \"kubernetes.io/projected/0756087c-fe35-4c07-b3de-961ead072e49-kube-api-access-2sshw\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:23.911418 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.911339    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15df66de-fd57-4bbd-873d-6e759ec3f367-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:23.913203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.913175    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15df66de-fd57-4bbd-873d-6e759ec3f367" (UID: "15df66de-fd57-4bbd-873d-6e759ec3f367"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:35:23.913203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.913190    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15df66de-fd57-4bbd-873d-6e759ec3f367" (UID: "15df66de-fd57-4bbd-873d-6e759ec3f367"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:35:23.913509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.913487    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15df66de-fd57-4bbd-873d-6e759ec3f367-kube-api-access-h8jps" (OuterVolumeSpecName: "kube-api-access-h8jps") pod "15df66de-fd57-4bbd-873d-6e759ec3f367" (UID: "15df66de-fd57-4bbd-873d-6e759ec3f367"). InnerVolumeSpecName "kube-api-access-h8jps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:35:23.917039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:23.917017    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89a0b8e7-bb2d-428a-876a-fbbefdf8ba06" path="/var/lib/kubelet/pods/89a0b8e7-bb2d-428a-876a-fbbefdf8ba06/volumes"
Sep 11 18:35:24.007647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.007619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:24.011801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.011779    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:24.011877 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.011803    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8jps\" (UniqueName: \"kubernetes.io/projected/15df66de-fd57-4bbd-873d-6e759ec3f367-kube-api-access-h8jps\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:24.011877 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.011817    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15df66de-fd57-4bbd-873d-6e759ec3f367-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:24.030104 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.030076    2563 generic.go:356] "Generic (PLEG): container finished" podID="0756087c-fe35-4c07-b3de-961ead072e49" containerID="97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857" exitCode=0
Sep 11 18:35:24.030195 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.030108    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh" event={"ID":"0756087c-fe35-4c07-b3de-961ead072e49","Type":"ContainerDied","Data":"97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857"}
Sep 11 18:35:24.030195 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.030139    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"
Sep 11 18:35:24.030195 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.030162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh" event={"ID":"0756087c-fe35-4c07-b3de-961ead072e49","Type":"ContainerDied","Data":"ed4d9582f7df7988190fec403c0811510a9644a8035a40bb0d7e14ed7e394a7b"}
Sep 11 18:35:24.030195 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.030191    2563 scope.go:117] "RemoveContainer" containerID="97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857"
Sep 11 18:35:24.032200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.032174    2563 generic.go:356] "Generic (PLEG): container finished" podID="15df66de-fd57-4bbd-873d-6e759ec3f367" containerID="e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae" exitCode=0
Sep 11 18:35:24.032378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.032227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" event={"ID":"15df66de-fd57-4bbd-873d-6e759ec3f367","Type":"ContainerDied","Data":"e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae"}
Sep 11 18:35:24.032378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.032281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7" event={"ID":"15df66de-fd57-4bbd-873d-6e759ec3f367","Type":"ContainerDied","Data":"7012268aa14700aef75921bc96e0873f805cf8d658e7eed863e4ff2175f25110"}
Sep 11 18:35:24.032378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.032289    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"
Sep 11 18:35:24.040323 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.040304    2563 scope.go:117] "RemoveContainer" containerID="97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857"
Sep 11 18:35:24.040656 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:24.040629    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857\": container with ID starting with 97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857 not found: ID does not exist" containerID="97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857"
Sep 11 18:35:24.040735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.040665    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857"} err="failed to get container status \"97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857\": rpc error: code = NotFound desc = could not find container \"97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857\": container with ID starting with 97062a1e01cee81b7c16b29ecd2485ca3bc050870ab2299bacb8f2b186ec7857 not found: ID does not exist"
Sep 11 18:35:24.040735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.040687    2563 scope.go:117] "RemoveContainer" containerID="e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae"
Sep 11 18:35:24.062447 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.062372    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"]
Sep 11 18:35:24.062876 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.062790    2563 scope.go:117] "RemoveContainer" containerID="e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae"
Sep 11 18:35:24.064151 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:24.063175    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae\": container with ID starting with e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae not found: ID does not exist" containerID="e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae"
Sep 11 18:35:24.064151 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.063233    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae"} err="failed to get container status \"e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae\": rpc error: code = NotFound desc = could not find container \"e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae\": container with ID starting with e472c4e9c5f8e30c78ae77f23dfea7edfef48ea0978e1a99594d374a43ae8aae not found: ID does not exist"
Sep 11 18:35:24.073919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.073890    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-d73031-6758784df9-fg9rh"]
Sep 11 18:35:24.082123 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.082094    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"]
Sep 11 18:35:24.087776 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.087744    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-d73031-dd9b675b8-9tcz7"]
Sep 11 18:35:24.842312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:24.842283    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:25.671851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:25.671815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:25.913575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:25.913535    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0756087c-fe35-4c07-b3de-961ead072e49" path="/var/lib/kubelet/pods/0756087c-fe35-4c07-b3de-961ead072e49/volumes"
Sep 11 18:35:25.914052 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:25.914031    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15df66de-fd57-4bbd-873d-6e759ec3f367" path="/var/lib/kubelet/pods/15df66de-fd57-4bbd-873d-6e759ec3f367/volumes"
Sep 11 18:35:26.555401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:26.555369    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:27.420910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.420877    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" podUID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" containerName="main" containerID="cri-o://70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8" gracePeriod=45
Sep 11 18:35:27.445302 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.445256    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn" podUID="4fd6f017-23ba-4144-bd33-e2ab511d2cb6" containerName="main" containerID="cri-o://c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9" gracePeriod=45
Sep 11 18:35:27.449397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.449372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:27.449521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.449464    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv" podUID="a1c08e9c-9742-45e1-b067-911205bc3a88" containerName="main" containerID="cri-o://4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9" gracePeriod=45
Sep 11 18:35:27.599563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.599522    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" podUID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" containerName="main" probeResult="failure" output="Get \"https://10.134.0.70:8000/ready\": dial tcp 10.134.0.70:8000: connect: connection refused"
Sep 11 18:35:27.759878 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.759854    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:35:27.762235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.762207    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:35:27.765627 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.765602    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:35:27.848415 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848387    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-home\") pod \"a1c08e9c-9742-45e1-b067-911205bc3a88\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rhfp\" (UniqueName: \"kubernetes.io/projected/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-kube-api-access-9rhfp\") pod \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848451    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-model-cache\") pod \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848474    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tls-certs\") pod \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848499    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-tmp-dir\") pod \"a1c08e9c-9742-45e1-b067-911205bc3a88\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848531    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c08e9c-9742-45e1-b067-911205bc3a88-tls-certs\") pod \"a1c08e9c-9742-45e1-b067-911205bc3a88\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-model-cache\") pod \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") "
Sep 11 18:35:27.848616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848618    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-home\") pod \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848650    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtgzl\" (UniqueName: \"kubernetes.io/projected/a1c08e9c-9742-45e1-b067-911205bc3a88-kube-api-access-mtgzl\") pod \"a1c08e9c-9742-45e1-b067-911205bc3a88\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848691    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tls-certs\") pod \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848730    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tmp-dir\") pod \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848729    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-model-cache" (OuterVolumeSpecName: "model-cache") pod "4fd6f017-23ba-4144-bd33-e2ab511d2cb6" (UID: "4fd6f017-23ba-4144-bd33-e2ab511d2cb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848781    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-home" (OuterVolumeSpecName: "home") pod "a1c08e9c-9742-45e1-b067-911205bc3a88" (UID: "a1c08e9c-9742-45e1-b067-911205bc3a88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848827    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-model-cache\") pod \"a1c08e9c-9742-45e1-b067-911205bc3a88\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848853    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-dshm\") pod \"a1c08e9c-9742-45e1-b067-911205bc3a88\" (UID: \"a1c08e9c-9742-45e1-b067-911205bc3a88\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848885    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-dshm\") pod \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvcf9\" (UniqueName: \"kubernetes.io/projected/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-kube-api-access-hvcf9\") pod \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\" (UID: \"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848951    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-home\") pod \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.848993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-dshm\") pod \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") "
Sep 11 18:35:27.849076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tmp-dir\") pod \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\" (UID: \"4fd6f017-23ba-4144-bd33-e2ab511d2cb6\") "
Sep 11 18:35:27.849822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849256    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-model-cache" (OuterVolumeSpecName: "model-cache") pod "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" (UID: "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.849822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1c08e9c-9742-45e1-b067-911205bc3a88" (UID: "a1c08e9c-9742-45e1-b067-911205bc3a88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.849822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849505    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4fd6f017-23ba-4144-bd33-e2ab511d2cb6" (UID: "4fd6f017-23ba-4144-bd33-e2ab511d2cb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.849822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849690    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" (UID: "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.849822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849802    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.849822 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849817    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.850146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849830    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.850146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849842    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.850146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849857    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.850146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849868    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.850146 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.849957    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-home" (OuterVolumeSpecName: "home") pod "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" (UID: "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.850435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.850157    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1c08e9c-9742-45e1-b067-911205bc3a88" (UID: "a1c08e9c-9742-45e1-b067-911205bc3a88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.850435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.850342    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-home" (OuterVolumeSpecName: "home") pod "4fd6f017-23ba-4144-bd33-e2ab511d2cb6" (UID: "4fd6f017-23ba-4144-bd33-e2ab511d2cb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.851645 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.851551    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4fd6f017-23ba-4144-bd33-e2ab511d2cb6" (UID: "4fd6f017-23ba-4144-bd33-e2ab511d2cb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:35:27.851645 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.851616    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" (UID: "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:35:27.851958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.851931    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-kube-api-access-9rhfp" (OuterVolumeSpecName: "kube-api-access-9rhfp") pod "4fd6f017-23ba-4144-bd33-e2ab511d2cb6" (UID: "4fd6f017-23ba-4144-bd33-e2ab511d2cb6"). InnerVolumeSpecName "kube-api-access-9rhfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:35:27.852389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.852360    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-kube-api-access-hvcf9" (OuterVolumeSpecName: "kube-api-access-hvcf9") pod "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" (UID: "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55"). InnerVolumeSpecName "kube-api-access-hvcf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:35:27.852595 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.852564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c08e9c-9742-45e1-b067-911205bc3a88-kube-api-access-mtgzl" (OuterVolumeSpecName: "kube-api-access-mtgzl") pod "a1c08e9c-9742-45e1-b067-911205bc3a88" (UID: "a1c08e9c-9742-45e1-b067-911205bc3a88"). InnerVolumeSpecName "kube-api-access-mtgzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:35:27.852782 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.852759    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-dshm" (OuterVolumeSpecName: "dshm") pod "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" (UID: "b7f7e4ff-eefa-4e72-9aa1-6057861c9a55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.852859 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.852821    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1c08e9c-9742-45e1-b067-911205bc3a88-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1c08e9c-9742-45e1-b067-911205bc3a88" (UID: "a1c08e9c-9742-45e1-b067-911205bc3a88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:35:27.852994 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.852981    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-dshm" (OuterVolumeSpecName: "dshm") pod "a1c08e9c-9742-45e1-b067-911205bc3a88" (UID: "a1c08e9c-9742-45e1-b067-911205bc3a88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.853803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.853788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-dshm" (OuterVolumeSpecName: "dshm") pod "4fd6f017-23ba-4144-bd33-e2ab511d2cb6" (UID: "4fd6f017-23ba-4144-bd33-e2ab511d2cb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:35:27.950741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950714    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950738    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1c08e9c-9742-45e1-b067-911205bc3a88-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950746    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950754    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvcf9\" (UniqueName: \"kubernetes.io/projected/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-kube-api-access-hvcf9\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950763    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950775    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950786    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rhfp\" (UniqueName: \"kubernetes.io/projected/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-kube-api-access-9rhfp\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950799    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950812    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1c08e9c-9742-45e1-b067-911205bc3a88-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950823    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950835    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtgzl\" (UniqueName: \"kubernetes.io/projected/a1c08e9c-9742-45e1-b067-911205bc3a88-kube-api-access-mtgzl\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:27.950919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:27.950847    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd6f017-23ba-4144-bd33-e2ab511d2cb6-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:35:28.058231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.058199    2563 generic.go:356] "Generic (PLEG): container finished" podID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" containerID="70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8" exitCode=0
Sep 11 18:35:28.058412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.058237    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"
Sep 11 18:35:28.058412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.058304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" event={"ID":"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55","Type":"ContainerDied","Data":"70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8"}
Sep 11 18:35:28.058412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.058343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc" event={"ID":"b7f7e4ff-eefa-4e72-9aa1-6057861c9a55","Type":"ContainerDied","Data":"92a3773c8e431a5a646031ab8cee0f9e8cd27a06cac93298b91f4ba71161767e"}
Sep 11 18:35:28.058412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.058363    2563 scope.go:117] "RemoveContainer" containerID="70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8"
Sep 11 18:35:28.059922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.059895    2563 generic.go:356] "Generic (PLEG): container finished" podID="4fd6f017-23ba-4144-bd33-e2ab511d2cb6" containerID="c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9" exitCode=0
Sep 11 18:35:28.060026 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.059997    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn" event={"ID":"4fd6f017-23ba-4144-bd33-e2ab511d2cb6","Type":"ContainerDied","Data":"c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9"}
Sep 11 18:35:28.060090 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.060038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn" event={"ID":"4fd6f017-23ba-4144-bd33-e2ab511d2cb6","Type":"ContainerDied","Data":"36730db426434e2a3b18d3975b9c861c3a1d24149b63f36208b6ee91998f4f8b"}
Sep 11 18:35:28.060215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.060180    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"
Sep 11 18:35:28.061786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.061707    2563 generic.go:356] "Generic (PLEG): container finished" podID="a1c08e9c-9742-45e1-b067-911205bc3a88" containerID="4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9" exitCode=0
Sep 11 18:35:28.061786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.061732    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv" event={"ID":"a1c08e9c-9742-45e1-b067-911205bc3a88","Type":"ContainerDied","Data":"4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9"}
Sep 11 18:35:28.061786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.061782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv" event={"ID":"a1c08e9c-9742-45e1-b067-911205bc3a88","Type":"ContainerDied","Data":"ad02a80eacd11326936a7e4f7381a6c86ec114172ad97a0f1e916fd6dd02e54e"}
Sep 11 18:35:28.062033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.061748    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"
Sep 11 18:35:28.069496 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.069479    2563 scope.go:117] "RemoveContainer" containerID="70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8"
Sep 11 18:35:28.069757 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:28.069740    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8\": container with ID starting with 70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8 not found: ID does not exist" containerID="70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8"
Sep 11 18:35:28.069805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.069765    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8"} err="failed to get container status \"70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8\": rpc error: code = NotFound desc = could not find container \"70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8\": container with ID starting with 70114a009344a611c35dd0796185342d8bf04b64681b8e6c70ca1e78b04ee2c8 not found: ID does not exist"
Sep 11 18:35:28.069805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.069786    2563 scope.go:117] "RemoveContainer" containerID="c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9"
Sep 11 18:35:28.077577 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.077561    2563 scope.go:117] "RemoveContainer" containerID="c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9"
Sep 11 18:35:28.077830 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:28.077808    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9\": container with ID starting with c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9 not found: ID does not exist" containerID="c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9"
Sep 11 18:35:28.077911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.077836    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9"} err="failed to get container status \"c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9\": rpc error: code = NotFound desc = could not find container \"c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9\": container with ID starting with c06f0fa4059019a3353e52759378467a271da0758fe458de6dd23523b2844dc9 not found: ID does not exist"
Sep 11 18:35:28.077911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.077855    2563 scope.go:117] "RemoveContainer" containerID="4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9"
Sep 11 18:35:28.082676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.082653    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"]
Sep 11 18:35:28.086339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.086316    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/simulator-w0-d73031-kserve-5d767f49cf-z72xv"]
Sep 11 18:35:28.087081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.087060    2563 scope.go:117] "RemoveContainer" containerID="4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9"
Sep 11 18:35:28.087347 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:28.087326    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9\": container with ID starting with 4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9 not found: ID does not exist" containerID="4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9"
Sep 11 18:35:28.087419 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.087356    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9"} err="failed to get container status \"4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9\": rpc error: code = NotFound desc = could not find container \"4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9\": container with ID starting with 4ab4a9eccd7d15f5a7d9d5d8cf141927697b6327a144e6f53a6a2c3daa79ecb9 not found: ID does not exist"
Sep 11 18:35:28.099348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.099328    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"]
Sep 11 18:35:28.103194 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.103170    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/premium-w0-d73031-kserve-9d9d9d98-jndcn"]
Sep 11 18:35:28.113747 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.113727    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"]
Sep 11 18:35:28.115687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.115669    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-d73031/distinct-2-w0-d73031-kserve-866984799d-6hlmc"]
Sep 11 18:35:28.195313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:28.195292    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:29.000967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:29.000938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:29.685913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:29.685887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:29.880229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:29.880188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:29.914836 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:29.914809    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fd6f017-23ba-4144-bd33-e2ab511d2cb6" path="/var/lib/kubelet/pods/4fd6f017-23ba-4144-bd33-e2ab511d2cb6/volumes"
Sep 11 18:35:29.915229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:29.915216    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1c08e9c-9742-45e1-b067-911205bc3a88" path="/var/lib/kubelet/pods/a1c08e9c-9742-45e1-b067-911205bc3a88/volumes"
Sep 11 18:35:29.915627 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:29.915614    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" path="/var/lib/kubelet/pods/b7f7e4ff-eefa-4e72-9aa1-6057861c9a55/volumes"
Sep 11 18:35:30.313098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.313071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:30.456754 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.456683    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:30.574914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.574833    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"]
Sep 11 18:35:30.575532 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575514    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15df66de-fd57-4bbd-873d-6e759ec3f367" containerName="maas-api"
Sep 11 18:35:30.575605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575535    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="15df66de-fd57-4bbd-873d-6e759ec3f367" containerName="maas-api"
Sep 11 18:35:30.575605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575564    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" containerName="main"
Sep 11 18:35:30.575605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575573    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" containerName="main"
Sep 11 18:35:30.575605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575588    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c08e9c-9742-45e1-b067-911205bc3a88" containerName="main"
Sep 11 18:35:30.575605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575597    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c08e9c-9742-45e1-b067-911205bc3a88" containerName="main"
Sep 11 18:35:30.575766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575624    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0756087c-fe35-4c07-b3de-961ead072e49" containerName="payload-processing"
Sep 11 18:35:30.575766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575632    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0756087c-fe35-4c07-b3de-961ead072e49" containerName="payload-processing"
Sep 11 18:35:30.575766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575644    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fd6f017-23ba-4144-bd33-e2ab511d2cb6" containerName="main"
Sep 11 18:35:30.575766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575652    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fd6f017-23ba-4144-bd33-e2ab511d2cb6" containerName="main"
Sep 11 18:35:30.575766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575749    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fd6f017-23ba-4144-bd33-e2ab511d2cb6" containerName="main"
Sep 11 18:35:30.575766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575763    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1c08e9c-9742-45e1-b067-911205bc3a88" containerName="main"
Sep 11 18:35:30.575948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575773    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0756087c-fe35-4c07-b3de-961ead072e49" containerName="payload-processing"
Sep 11 18:35:30.575948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575782    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="15df66de-fd57-4bbd-873d-6e759ec3f367" containerName="maas-api"
Sep 11 18:35:30.575948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.575790    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7f7e4ff-eefa-4e72-9aa1-6057861c9a55" containerName="main"
Sep 11 18:35:30.581033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.581012    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.583435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.583414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b77bc8-openshift-default-dockercfg-xvlrs\""
Sep 11 18:35:30.592203 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.592179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"]
Sep 11 18:35:30.614284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.609273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:30.676400 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-token\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676565 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istiod-ca-cert\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676565 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-credential-socket\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676699 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-envoy\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676699 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-certs\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676699 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-podinfo\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-socket\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c2tg\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-kube-api-access-2c2tg\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.676854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.676816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-data\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-token\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istiod-ca-cert\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-credential-socket\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777453    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-envoy\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-certs\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-podinfo\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777586    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-socket\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c2tg\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-kube-api-access-2c2tg\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.777883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777659    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-data\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.778015 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.777985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-credential-socket\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.778075 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.778034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-socket\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.778249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.778219    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-data\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.778378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.778245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-certs\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.778534 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.778511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istiod-ca-cert\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.780176 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.780155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-envoy\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.780295 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.780179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-podinfo\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.785759 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.785738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-token\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.785858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.785793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c2tg\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-kube-api-access-2c2tg\") pod \"e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") " pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.895073 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.894948    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:30.916036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:30.916009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:31.173452 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.173419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"]
Sep 11 18:35:31.175634 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:35:31.175605    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b49cfa6_71cc_4f8a_9ac2_744518df92c4.slice/crio-217315a1dc23edb40f578c7749eed55ef97e8468a4744cfc891bec82225730c0 WatchSource:0}: Error finding container 217315a1dc23edb40f578c7749eed55ef97e8468a4744cfc891bec82225730c0: Status 404 returned error can't find the container with id 217315a1dc23edb40f578c7749eed55ef97e8468a4744cfc891bec82225730c0
Sep 11 18:35:31.177948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.177912    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:31.178028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.177981    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:31.178028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.178011    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:31.238601 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.238569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:31.338908 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.338876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:31.487550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.487480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:31.982781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.982755    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:35:31.986818 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:31.986797    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:35:32.088239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.088211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" event={"ID":"7b49cfa6-71cc-4f8a-9ac2-744518df92c4","Type":"ContainerStarted","Data":"3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"}
Sep 11 18:35:32.088382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.088285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" event={"ID":"7b49cfa6-71cc-4f8a-9ac2-744518df92c4","Type":"ContainerStarted","Data":"217315a1dc23edb40f578c7749eed55ef97e8468a4744cfc891bec82225730c0"}
Sep 11 18:35:32.089613 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.089587    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:32.109963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.109924    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podStartSLOduration=2.109910095 podStartE2EDuration="2.109910095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:32.108221263 +0000 UTC m=+1500.701421821" watchObservedRunningTime="2026-09-11 18:35:32.109910095 +0000 UTC m=+1500.703110651"
Sep 11 18:35:32.895999 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.895966    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:32.901179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.901158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:32.959923 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:32.959897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:33.093848 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:33.093814    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:33.094846 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:33.094822    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:35:33.680276 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:33.680228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:34.047062 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:34.046999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:34.139131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:34.139101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:34.540095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:34.540068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:35.595851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:35.595816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:36.301654 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:36.301623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:36.994191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:36.994158    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:37.263847 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:37.263763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:37.322112 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:37.322084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:37.806175 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:37.806145    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:38.482816 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:38.482782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:39.280237 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.280203    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"]
Sep 11 18:35:39.284466 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.284440    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.286638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.286604    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b77bc8-serving-cert\""
Sep 11 18:35:39.294526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.294495    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"]
Sep 11 18:35:39.306131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.306102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:39.366394 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.366358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:39.376796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.376767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8430aae-66f8-46bd-a6ea-ed9503951939-openshift-service-ca\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.376874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.376812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.376910 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.376894    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbngd\" (UniqueName: \"kubernetes.io/projected/a8430aae-66f8-46bd-a6ea-ed9503951939-kube-api-access-bbngd\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.376997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.376984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-metrics-certs\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.478326 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.478287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8430aae-66f8-46bd-a6ea-ed9503951939-openshift-service-ca\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.478507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.478343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.478507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.478412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbngd\" (UniqueName: \"kubernetes.io/projected/a8430aae-66f8-46bd-a6ea-ed9503951939-kube-api-access-bbngd\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.478507 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:39.478460    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-serving-cert: secret "maas-api-e2e-api-b-b77bc8-serving-cert" not found
Sep 11 18:35:39.478677 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.478516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-metrics-certs\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.478677 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:35:39.478541    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls podName:a8430aae-66f8-46bd-a6ea-ed9503951939 nodeName:}" failed. No retries permitted until 2026-09-11 18:35:39.978521644 +0000 UTC m=+1508.571722192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls") pod "maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" (UID: "a8430aae-66f8-46bd-a6ea-ed9503951939") : secret "maas-api-e2e-api-b-b77bc8-serving-cert" not found
Sep 11 18:35:39.478954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.478935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8430aae-66f8-46bd-a6ea-ed9503951939-openshift-service-ca\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.480960 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.480937    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-metrics-certs\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.486962 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.486931    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbngd\" (UniqueName: \"kubernetes.io/projected/a8430aae-66f8-46bd-a6ea-ed9503951939-kube-api-access-bbngd\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.608242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.608177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:39.890417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.890329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:39.982335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.982302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.984713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.984691    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls\") pod \"maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:39.989535 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:39.989504    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:40.018486 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.018460    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"]
Sep 11 18:35:40.022377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.022362    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.024624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.024596    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b77bc8\""
Sep 11 18:35:40.024724 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.024699    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b77bc8-dockercfg-bkf7m\""
Sep 11 18:35:40.031412 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.031391    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"]
Sep 11 18:35:40.071196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.071166    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"]
Sep 11 18:35:40.074901 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.074878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.083526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.083500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf5fj\" (UniqueName: \"kubernetes.io/projected/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-kube-api-access-lf5fj\") pod \"payload-processing-e2e-api-b-b77bc8-75675867c9-6shms\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.083691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.083669    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b77bc8-75675867c9-6shms\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.085660 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.085636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"]
Sep 11 18:35:40.185244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.185159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65xtj\" (UniqueName: \"kubernetes.io/projected/748e1efb-e270-4e97-9006-b59f9aab0dee-kube-api-access-65xtj\") pod \"payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.185244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.185203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf5fj\" (UniqueName: \"kubernetes.io/projected/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-kube-api-access-lf5fj\") pod \"payload-processing-e2e-api-b-b77bc8-75675867c9-6shms\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.185509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.185313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/748e1efb-e270-4e97-9006-b59f9aab0dee-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.185509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.185445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b77bc8-75675867c9-6shms\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.186132 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.186115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-plugins-config-volume\") pod \"payload-processing-e2e-api-b-b77bc8-75675867c9-6shms\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.193488 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.193461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf5fj\" (UniqueName: \"kubernetes.io/projected/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-kube-api-access-lf5fj\") pod \"payload-processing-e2e-api-b-b77bc8-75675867c9-6shms\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") " pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.197126 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.197105    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:40.318372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.316762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65xtj\" (UniqueName: \"kubernetes.io/projected/748e1efb-e270-4e97-9006-b59f9aab0dee-kube-api-access-65xtj\") pod \"payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.318372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.316900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/748e1efb-e270-4e97-9006-b59f9aab0dee-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.318372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.318153    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/748e1efb-e270-4e97-9006-b59f9aab0dee-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.336952 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.335665    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:40.353336 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.348979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65xtj\" (UniqueName: \"kubernetes.io/projected/748e1efb-e270-4e97-9006-b59f9aab0dee-kube-api-access-65xtj\") pod \"payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.390191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.390124    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:40.495159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.495123    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"]
Sep 11 18:35:40.495746 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:35:40.495676    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8430aae_66f8_46bd_a6ea_ed9503951939.slice/crio-60cf98c810f357f25ed424e1cabadf4e23226f5094248105dc20cb429791e7c6 WatchSource:0}: Error finding container 60cf98c810f357f25ed424e1cabadf4e23226f5094248105dc20cb429791e7c6: Status 404 returned error can't find the container with id 60cf98c810f357f25ed424e1cabadf4e23226f5094248105dc20cb429791e7c6
Sep 11 18:35:40.507281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.498040    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:35:40.661943 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.661914    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"]
Sep 11 18:35:40.663764 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:35:40.663737    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71a7123_db02_4e9e_b2fc_b3a2a97edce6.slice/crio-86aaff26fb7524a2f846664b922b5f36dd2a3b2062f8eb1b553dbe53e6095786 WatchSource:0}: Error finding container 86aaff26fb7524a2f846664b922b5f36dd2a3b2062f8eb1b553dbe53e6095786: Status 404 returned error can't find the container with id 86aaff26fb7524a2f846664b922b5f36dd2a3b2062f8eb1b553dbe53e6095786
Sep 11 18:35:40.695306 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:40.695285    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"]
Sep 11 18:35:40.724777 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:35:40.724741    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748e1efb_e270_4e97_9006_b59f9aab0dee.slice/crio-985c956b4e52c35e6b738ac9f106d41fc16ace1e2354e51b9325865c738842a7 WatchSource:0}: Error finding container 985c956b4e52c35e6b738ac9f106d41fc16ace1e2354e51b9325865c738842a7: Status 404 returned error can't find the container with id 985c956b4e52c35e6b738ac9f106d41fc16ace1e2354e51b9325865c738842a7
Sep 11 18:35:41.160352 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.160319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms" event={"ID":"a71a7123-db02-4e9e-b2fc-b3a2a97edce6","Type":"ContainerStarted","Data":"d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1"}
Sep 11 18:35:41.160519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.160377    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms" event={"ID":"a71a7123-db02-4e9e-b2fc-b3a2a97edce6","Type":"ContainerStarted","Data":"86aaff26fb7524a2f846664b922b5f36dd2a3b2062f8eb1b553dbe53e6095786"}
Sep 11 18:35:41.160519 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.160418    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:41.161996 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.161973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" event={"ID":"a8430aae-66f8-46bd-a6ea-ed9503951939","Type":"ContainerStarted","Data":"12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f"}
Sep 11 18:35:41.162111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.162020    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:41.162111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.162042    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" event={"ID":"a8430aae-66f8-46bd-a6ea-ed9503951939","Type":"ContainerStarted","Data":"60cf98c810f357f25ed424e1cabadf4e23226f5094248105dc20cb429791e7c6"}
Sep 11 18:35:41.163671 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.163649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw" event={"ID":"748e1efb-e270-4e97-9006-b59f9aab0dee","Type":"ContainerStarted","Data":"fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab"}
Sep 11 18:35:41.163766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.163686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw" event={"ID":"748e1efb-e270-4e97-9006-b59f9aab0dee","Type":"ContainerStarted","Data":"985c956b4e52c35e6b738ac9f106d41fc16ace1e2354e51b9325865c738842a7"}
Sep 11 18:35:41.163766 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.163715    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:41.177937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.177901    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms" podStartSLOduration=2.177889424 podStartE2EDuration="2.177889424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:41.176949186 +0000 UTC m=+1509.770149735" watchObservedRunningTime="2026-09-11 18:35:41.177889424 +0000 UTC m=+1509.771089981"
Sep 11 18:35:41.195478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.195439    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" podStartSLOduration=1.775054671 podStartE2EDuration="2.19542762s" totalImagesPullingTime="420.372949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:41.193646535 +0000 UTC m=+1509.786847093" watchObservedRunningTime="2026-09-11 18:35:41.19542762 +0000 UTC m=+1509.788628176"
Sep 11 18:35:41.212479 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:41.212388    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw" podStartSLOduration=1.212376018 podStartE2EDuration="1.212376018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:35:41.209510682 +0000 UTC m=+1509.802711240" watchObservedRunningTime="2026-09-11 18:35:41.212376018 +0000 UTC m=+1509.805576576"
Sep 11 18:35:47.176208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:47.176174    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:35:48.111907 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:48.111876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:48.902149 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:48.902115    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:48.902566 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:48.902207    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:49.213989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.213841    2563 generic.go:356] "Generic (PLEG): container finished" podID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerID="cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e" exitCode=0
Sep 11 18:35:49.213989 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.213885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" event={"ID":"4c2d2943-36ae-43df-b7e4-bf5fb239049c","Type":"ContainerDied","Data":"cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"}
Sep 11 18:35:49.214711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.214673    2563 scope.go:117] "RemoveContainer" containerID="cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"
Sep 11 18:35:49.216408 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.216344    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:49.216569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.216473    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:49.216569 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.216521    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:35:49.249400 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.249367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:49.309098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:49.309073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:50.122384 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:50.122343    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:50.226401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:50.226358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" event={"ID":"4c2d2943-36ae-43df-b7e4-bf5fb239049c","Type":"ContainerStarted","Data":"2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30"}
Sep 11 18:35:50.245641 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:50.245616    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:50.245906 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:50.245884    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:50.245956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:50.245930    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:51.245990 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:51.245956    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:51.246508 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:51.246015    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:52.171919 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:52.171878    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:35:52.172946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:52.172923    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:35:52.246419 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:52.246382    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:52.246780 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:52.246436    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:53.246102 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:53.246070    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:53.246295 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:53.246123    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:54.245630 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:54.245550    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:54.245630 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:54.245613    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:55.245616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:55.245571    2563 patch_prober.go:28] interesting pod/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 11 18:35:55.245789 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:55.245654    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 11 18:35:56.054833 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:56.054782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:35:56.250033 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:56.249998    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:56.250439 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:56.250255    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:35:56.250959 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:35:56.250939    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:36:00.512425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:00.512324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:01.958117 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:01.958081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:02.691329 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:02.691285    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:03.400683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:03.400649    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:04.123720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:04.123650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:04.135440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:04.135413    2563 patch_prober.go:28] interesting pod/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 18:36:04.135558 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:04.135458    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 18:36:04.223844 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:04.223815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:05.046896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.046825    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:05.317413 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.317383    2563 generic.go:356] "Generic (PLEG): container finished" podID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerID="3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995" exitCode=0
Sep 11 18:36:05.317575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.317452    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" event={"ID":"7b49cfa6-71cc-4f8a-9ac2-744518df92c4","Type":"ContainerDied","Data":"3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"}
Sep 11 18:36:05.317900 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.317887    2563 scope.go:117] "RemoveContainer" containerID="3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"
Sep 11 18:36:05.319142 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.319112    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:36:05.319230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.319179    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:36:05.319230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.319210    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:36:05.791503 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:05.791455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:06.328122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:06.328084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" event={"ID":"7b49cfa6-71cc-4f8a-9ac2-744518df92c4","Type":"ContainerStarted","Data":"d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e"}
Sep 11 18:36:06.639174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:06.639083    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:06.896128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:06.896048    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:36:06.896327 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:06.896301    2563 patch_prober.go:28] interesting pod/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 11 18:36:06.896386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:06.896367    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 11 18:36:07.715696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:07.715654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:07.895949 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:07.895914    2563 patch_prober.go:28] interesting pod/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 11 18:36:07.896107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:07.895968    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 11 18:36:08.445002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:08.444970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:08.895866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:08.895823    2563 patch_prober.go:28] interesting pod/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 11 18:36:08.896286 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:08.895890    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 11 18:36:09.524746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:09.524696    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:09.895959 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:09.895923    2563 patch_prober.go:28] interesting pod/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 11 18:36:09.896383 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:09.895975    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 11 18:36:10.276926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:10.276894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:10.895497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:10.895460    2563 patch_prober.go:28] interesting pod/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Sep 11 18:36:10.895657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:10.895524    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Sep 11 18:36:11.613115 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:11.613084    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:11.899455 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:11.899384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:36:11.899850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:11.899831    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:36:11.900643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:11.900613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:36:11.900746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:11.900700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:12.613021 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:12.612991    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:13.397459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:13.397427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:17.491969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:17.491934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:23.132950 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.132912    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:23.172874 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.172844    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"]
Sep 11 18:36:23.173064 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.173043    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb" podUID="d9e8bd2f-899c-4378-8172-4d5f6a0e3634" containerName="payload-pre-processing" containerID="cri-o://7fc2f5b5d3d2582eebb86c83ff43df4c05eefe9b15ebea0f359c61ad1dba7ceb" gracePeriod=30
Sep 11 18:36:23.234052 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.234018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:23.336841 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.336810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:23.432266 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.432237    2563 generic.go:356] "Generic (PLEG): container finished" podID="d9e8bd2f-899c-4378-8172-4d5f6a0e3634" containerID="7fc2f5b5d3d2582eebb86c83ff43df4c05eefe9b15ebea0f359c61ad1dba7ceb" exitCode=0
Sep 11 18:36:23.432408 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.432328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb" event={"ID":"d9e8bd2f-899c-4378-8172-4d5f6a0e3634","Type":"ContainerDied","Data":"7fc2f5b5d3d2582eebb86c83ff43df4c05eefe9b15ebea0f359c61ad1dba7ceb"}
Sep 11 18:36:23.432483 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.432420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb" event={"ID":"d9e8bd2f-899c-4378-8172-4d5f6a0e3634","Type":"ContainerDied","Data":"3a3ca4dade46788a4b51d4b1545191e44be600e8f8e65b5a05cc65f45b3d5150"}
Sep 11 18:36:23.432553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.432490    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a3ca4dade46788a4b51d4b1545191e44be600e8f8e65b5a05cc65f45b3d5150"
Sep 11 18:36:23.433420 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.433396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:23.438781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.438763    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:36:23.482829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.482800    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8m8k\" (UniqueName: \"kubernetes.io/projected/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-kube-api-access-d8m8k\") pod \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") "
Sep 11 18:36:23.482979 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.482940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-plugins-config-volume\") pod \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\" (UID: \"d9e8bd2f-899c-4378-8172-4d5f6a0e3634\") "
Sep 11 18:36:23.483373 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.483348    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9e8bd2f-899c-4378-8172-4d5f6a0e3634" (UID: "d9e8bd2f-899c-4378-8172-4d5f6a0e3634"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:36:23.484926 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.484904    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-kube-api-access-d8m8k" (OuterVolumeSpecName: "kube-api-access-d8m8k") pod "d9e8bd2f-899c-4378-8172-4d5f6a0e3634" (UID: "d9e8bd2f-899c-4378-8172-4d5f6a0e3634"). InnerVolumeSpecName "kube-api-access-d8m8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:23.584765 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.584722    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8m8k\" (UniqueName: \"kubernetes.io/projected/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-kube-api-access-d8m8k\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:23.584765 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.584758    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9e8bd2f-899c-4378-8172-4d5f6a0e3634-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:23.702035 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.701944    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:23.815991 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:23.815956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:24.438406 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:24.438375    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"
Sep 11 18:36:24.457273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:24.457230    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"]
Sep 11 18:36:24.461118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:24.461090    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1e370a-5bc569f9cd-2w9pb"]
Sep 11 18:36:24.619400 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:24.619372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:25.442854 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:25.442826    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:25.913315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:25.913286    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e8bd2f-899c-4378-8172-4d5f6a0e3634" path="/var/lib/kubelet/pods/d9e8bd2f-899c-4378-8172-4d5f6a0e3634/volumes"
Sep 11 18:36:26.114219 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:26.114189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:26.943068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:26.943034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:27.614577 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:27.614539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:28.401967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:28.401929    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:30.235346 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:30.235316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:30.336421 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:30.336391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:30.486729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:30.486652    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"]
Sep 11 18:36:30.486908 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:30.486870    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy" containerID="cri-o://2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30" gracePeriod=30
Sep 11 18:36:30.511636 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:30.511612    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:30.998718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:30.998680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:31.784236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:31.784201    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:32.110150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:32.110080    2563 scope.go:117] "RemoveContainer" containerID="3a9d89e56c4e5679da81ce766eb1f046a436073191a7f059d44cc92a163fb28b"
Sep 11 18:36:32.580397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:32.580358    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:33.285973 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:33.285941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:33.970888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:33.970855    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:34.755127 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:34.755095    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:35.685702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.685674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:35.750993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.750929    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:36:35.810324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-data\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810343    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-762xt\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-kube-api-access-762xt\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810390    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-podinfo\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810421    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istiod-ca-cert\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810451    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-envoy\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-socket\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810567    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-token\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810570    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:35.810692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810626    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-certs\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.811107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810784    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-credential-socket\") pod \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\" (UID: \"4c2d2943-36ae-43df-b7e4-bf5fb239049c\") "
Sep 11 18:36:35.811107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810795    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:35.811107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810841    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:35.811107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.810857    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:36:35.811107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.811057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:35.811393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.811249    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.811393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.811299    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.811393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.811314    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.811393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.811328    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.811393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.811342    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.812814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.812788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:35.812945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.812866    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:36:35.813201 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.813181    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-kube-api-access-762xt" (OuterVolumeSpecName: "kube-api-access-762xt") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "kube-api-access-762xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:35.813315 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.813300    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c2d2943-36ae-43df-b7e4-bf5fb239049c" (UID: "4c2d2943-36ae-43df-b7e4-bf5fb239049c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:35.912600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.912579    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.912600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.912598    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.912755 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.912616    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:35.912755 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:35.912626    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-762xt\" (UniqueName: \"kubernetes.io/projected/4c2d2943-36ae-43df-b7e4-bf5fb239049c-kube-api-access-762xt\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:36.349173 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.349138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:36.509494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.509458    2563 generic.go:356] "Generic (PLEG): container finished" podID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerID="2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30" exitCode=0
Sep 11 18:36:36.509494 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.509497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" event={"ID":"4c2d2943-36ae-43df-b7e4-bf5fb239049c","Type":"ContainerDied","Data":"2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30"}
Sep 11 18:36:36.509683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.509504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"
Sep 11 18:36:36.509683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.509540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5" event={"ID":"4c2d2943-36ae-43df-b7e4-bf5fb239049c","Type":"ContainerDied","Data":"182f41733ae53efbe8f5ab40c50c8943689ad671b2321f4aec6afebb1d4fc7b4"}
Sep 11 18:36:36.509683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.509562    2563 scope.go:117] "RemoveContainer" containerID="2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30"
Sep 11 18:36:36.526404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.526384    2563 scope.go:117] "RemoveContainer" containerID="cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"
Sep 11 18:36:36.533104 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.533026    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"]
Sep 11 18:36:36.536689 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.536663    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1e370a-openshift-default-6fc4f7d86b-hcqf5"]
Sep 11 18:36:36.537474 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.537459    2563 scope.go:117] "RemoveContainer" containerID="2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30"
Sep 11 18:36:36.537754 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:36.537734    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30\": container with ID starting with 2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30 not found: ID does not exist" containerID="2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30"
Sep 11 18:36:36.537803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.537760    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30"} err="failed to get container status \"2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30\": rpc error: code = NotFound desc = could not find container \"2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30\": container with ID starting with 2da1f19b063c79a90e3fb8e480265119734494fbffc793e6a272c2df45b68b30 not found: ID does not exist"
Sep 11 18:36:36.537803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.537776    2563 scope.go:117] "RemoveContainer" containerID="cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"
Sep 11 18:36:36.538037 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:36.538017    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e\": container with ID starting with cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e not found: ID does not exist" containerID="cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"
Sep 11 18:36:36.538098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:36.538046    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e"} err="failed to get container status \"cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e\": rpc error: code = NotFound desc = could not find container \"cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e\": container with ID starting with cf60c24a01de4013f688945afcc0814f811598c8e127f6da2672152060eff77e not found: ID does not exist"
Sep 11 18:36:37.339374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:37.339340    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:37.872718 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:37.872682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:37.914098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:37.914062    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" path="/var/lib/kubelet/pods/4c2d2943-36ae-43df-b7e4-bf5fb239049c/volumes"
Sep 11 18:36:38.690985 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:38.690953    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:39.360088 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:39.360050    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:40.097244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:40.097208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:40.773866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:40.773820    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:41.500378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:41.500347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:42.188135 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:42.188099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:42.350823 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:42.350790    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"]
Sep 11 18:36:42.436671 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:42.436644    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:42.989417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:42.989386    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:43.947244 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:43.947209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:48.589340 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.589291    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"]
Sep 11 18:36:48.589832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.589561    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" podUID="a8430aae-66f8-46bd-a6ea-ed9503951939" containerName="maas-api" containerID="cri-o://12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f" gracePeriod=30
Sep 11 18:36:48.615462 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.615423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:48.647756 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.647724    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"]
Sep 11 18:36:48.648125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.647985    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms" podUID="a71a7123-db02-4e9e-b2fc-b3a2a97edce6" containerName="payload-processing" containerID="cri-o://d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1" gracePeriod=30
Sep 11 18:36:48.658313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.658232    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"]
Sep 11 18:36:48.658526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.658461    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw" podUID="748e1efb-e270-4e97-9006-b59f9aab0dee" containerName="payload-pre-processing" containerID="cri-o://fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab" gracePeriod=30
Sep 11 18:36:48.690850 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.690819    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:48.788723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.788682    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:48.847712 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.847637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:48.933436 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.933025    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:36:48.948147 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.948122    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:36:48.958064 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:48.957717    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:36:49.040236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040202    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8430aae-66f8-46bd-a6ea-ed9503951939-openshift-service-ca\") pod \"a8430aae-66f8-46bd-a6ea-ed9503951939\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") "
Sep 11 18:36:49.040529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbngd\" (UniqueName: \"kubernetes.io/projected/a8430aae-66f8-46bd-a6ea-ed9503951939-kube-api-access-bbngd\") pod \"a8430aae-66f8-46bd-a6ea-ed9503951939\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") "
Sep 11 18:36:49.040702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040625    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65xtj\" (UniqueName: \"kubernetes.io/projected/748e1efb-e270-4e97-9006-b59f9aab0dee-kube-api-access-65xtj\") pod \"748e1efb-e270-4e97-9006-b59f9aab0dee\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") "
Sep 11 18:36:49.040702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040663    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/748e1efb-e270-4e97-9006-b59f9aab0dee-plugins-config-volume\") pod \"748e1efb-e270-4e97-9006-b59f9aab0dee\" (UID: \"748e1efb-e270-4e97-9006-b59f9aab0dee\") "
Sep 11 18:36:49.040702 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-plugins-config-volume\") pod \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") "
Sep 11 18:36:49.040953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040753    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls\") pod \"a8430aae-66f8-46bd-a6ea-ed9503951939\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") "
Sep 11 18:36:49.040953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-metrics-certs\") pod \"a8430aae-66f8-46bd-a6ea-ed9503951939\" (UID: \"a8430aae-66f8-46bd-a6ea-ed9503951939\") "
Sep 11 18:36:49.041893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.040656    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8430aae-66f8-46bd-a6ea-ed9503951939-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a8430aae-66f8-46bd-a6ea-ed9503951939" (UID: "a8430aae-66f8-46bd-a6ea-ed9503951939"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:36:49.041893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.041021    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/748e1efb-e270-4e97-9006-b59f9aab0dee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "748e1efb-e270-4e97-9006-b59f9aab0dee" (UID: "748e1efb-e270-4e97-9006-b59f9aab0dee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:36:49.041893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.041856    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a71a7123-db02-4e9e-b2fc-b3a2a97edce6" (UID: "a71a7123-db02-4e9e-b2fc-b3a2a97edce6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:36:49.043281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.043226    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748e1efb-e270-4e97-9006-b59f9aab0dee-kube-api-access-65xtj" (OuterVolumeSpecName: "kube-api-access-65xtj") pod "748e1efb-e270-4e97-9006-b59f9aab0dee" (UID: "748e1efb-e270-4e97-9006-b59f9aab0dee"). InnerVolumeSpecName "kube-api-access-65xtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:49.043606 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.043565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a8430aae-66f8-46bd-a6ea-ed9503951939" (UID: "a8430aae-66f8-46bd-a6ea-ed9503951939"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:36:49.043696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.043670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a8430aae-66f8-46bd-a6ea-ed9503951939" (UID: "a8430aae-66f8-46bd-a6ea-ed9503951939"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:36:49.043975 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.043958    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8430aae-66f8-46bd-a6ea-ed9503951939-kube-api-access-bbngd" (OuterVolumeSpecName: "kube-api-access-bbngd") pod "a8430aae-66f8-46bd-a6ea-ed9503951939" (UID: "a8430aae-66f8-46bd-a6ea-ed9503951939"). InnerVolumeSpecName "kube-api-access-bbngd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:49.141730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf5fj\" (UniqueName: \"kubernetes.io/projected/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-kube-api-access-lf5fj\") pod \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\" (UID: \"a71a7123-db02-4e9e-b2fc-b3a2a97edce6\") "
Sep 11 18:36:49.141888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141845    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8430aae-66f8-46bd-a6ea-ed9503951939-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.141888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141857    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbngd\" (UniqueName: \"kubernetes.io/projected/a8430aae-66f8-46bd-a6ea-ed9503951939-kube-api-access-bbngd\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.141888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141868    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65xtj\" (UniqueName: \"kubernetes.io/projected/748e1efb-e270-4e97-9006-b59f9aab0dee-kube-api-access-65xtj\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.141888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141876    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/748e1efb-e270-4e97-9006-b59f9aab0dee-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.141888 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141886    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.142162 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141895    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.142162 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.141904    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8430aae-66f8-46bd-a6ea-ed9503951939-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.143652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.143621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-kube-api-access-lf5fj" (OuterVolumeSpecName: "kube-api-access-lf5fj") pod "a71a7123-db02-4e9e-b2fc-b3a2a97edce6" (UID: "a71a7123-db02-4e9e-b2fc-b3a2a97edce6"). InnerVolumeSpecName "kube-api-access-lf5fj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:49.179273 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.179232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:49.243051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.243024    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf5fj\" (UniqueName: \"kubernetes.io/projected/a71a7123-db02-4e9e-b2fc-b3a2a97edce6-kube-api-access-lf5fj\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:49.375507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.375472    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:49.595484 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.595450    2563 generic.go:356] "Generic (PLEG): container finished" podID="748e1efb-e270-4e97-9006-b59f9aab0dee" containerID="fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab" exitCode=0
Sep 11 18:36:49.595921 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.595494    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"
Sep 11 18:36:49.595921 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.595535    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw" event={"ID":"748e1efb-e270-4e97-9006-b59f9aab0dee","Type":"ContainerDied","Data":"fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab"}
Sep 11 18:36:49.595921 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.595603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw" event={"ID":"748e1efb-e270-4e97-9006-b59f9aab0dee","Type":"ContainerDied","Data":"985c956b4e52c35e6b738ac9f106d41fc16ace1e2354e51b9325865c738842a7"}
Sep 11 18:36:49.595921 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.595637    2563 scope.go:117] "RemoveContainer" containerID="fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab"
Sep 11 18:36:49.596965 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.596952    2563 generic.go:356] "Generic (PLEG): container finished" podID="a71a7123-db02-4e9e-b2fc-b3a2a97edce6" containerID="d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1" exitCode=0
Sep 11 18:36:49.597043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.596999    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"
Sep 11 18:36:49.597098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.597031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms" event={"ID":"a71a7123-db02-4e9e-b2fc-b3a2a97edce6","Type":"ContainerDied","Data":"d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1"}
Sep 11 18:36:49.597098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.597085    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms" event={"ID":"a71a7123-db02-4e9e-b2fc-b3a2a97edce6","Type":"ContainerDied","Data":"86aaff26fb7524a2f846664b922b5f36dd2a3b2062f8eb1b553dbe53e6095786"}
Sep 11 18:36:49.600012 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.598479    2563 generic.go:356] "Generic (PLEG): container finished" podID="a8430aae-66f8-46bd-a6ea-ed9503951939" containerID="12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f" exitCode=0
Sep 11 18:36:49.600012 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.598516    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"
Sep 11 18:36:49.600012 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.598544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" event={"ID":"a8430aae-66f8-46bd-a6ea-ed9503951939","Type":"ContainerDied","Data":"12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f"}
Sep 11 18:36:49.600012 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.598583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf" event={"ID":"a8430aae-66f8-46bd-a6ea-ed9503951939","Type":"ContainerDied","Data":"60cf98c810f357f25ed424e1cabadf4e23226f5094248105dc20cb429791e7c6"}
Sep 11 18:36:49.621404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.621385    2563 scope.go:117] "RemoveContainer" containerID="fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab"
Sep 11 18:36:49.621676 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:49.621650    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab\": container with ID starting with fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab not found: ID does not exist" containerID="fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab"
Sep 11 18:36:49.621776 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.621686    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab"} err="failed to get container status \"fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab\": rpc error: code = NotFound desc = could not find container \"fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab\": container with ID starting with fd322696512b536752b560eb502aa7be5f87f2962ff69ca0ec6c1a8a5d60b5ab not found: ID does not exist"
Sep 11 18:36:49.621776 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.621708    2563 scope.go:117] "RemoveContainer" containerID="d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1"
Sep 11 18:36:49.630060 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.630037    2563 scope.go:117] "RemoveContainer" containerID="d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1"
Sep 11 18:36:49.630335 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:49.630315    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1\": container with ID starting with d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1 not found: ID does not exist" containerID="d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1"
Sep 11 18:36:49.630426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.630344    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1"} err="failed to get container status \"d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1\": rpc error: code = NotFound desc = could not find container \"d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1\": container with ID starting with d8f87a8b7b8a2b626fecddb2b67f50a0c5ea814ad17900b7a73e98ac424a01b1 not found: ID does not exist"
Sep 11 18:36:49.630426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.630365    2563 scope.go:117] "RemoveContainer" containerID="12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f"
Sep 11 18:36:49.637526 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.637505    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"]
Sep 11 18:36:49.638953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.638932    2563 scope.go:117] "RemoveContainer" containerID="12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f"
Sep 11 18:36:49.639319 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:49.639229    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f\": container with ID starting with 12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f not found: ID does not exist" containerID="12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f"
Sep 11 18:36:49.639319 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.639274    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f"} err="failed to get container status \"12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f\": rpc error: code = NotFound desc = could not find container \"12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f\": container with ID starting with 12c63c076171f1e821b7db45f32c5350e8c6392c1a137d698c7df08f7a35611f not found: ID does not exist"
Sep 11 18:36:49.641478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.641450    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b77bc8-6c8d5cc5fb-r87rf"]
Sep 11 18:36:49.651285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.651250    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"]
Sep 11 18:36:49.655221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.655203    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b77bc8-74b5fdc466-nm8hw"]
Sep 11 18:36:49.665804 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.665786    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"]
Sep 11 18:36:49.668286 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.668251    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-b77bc8-75675867c9-6shms"]
Sep 11 18:36:49.913317 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.913214    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="748e1efb-e270-4e97-9006-b59f9aab0dee" path="/var/lib/kubelet/pods/748e1efb-e270-4e97-9006-b59f9aab0dee/volumes"
Sep 11 18:36:49.913692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.913676    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71a7123-db02-4e9e-b2fc-b3a2a97edce6" path="/var/lib/kubelet/pods/a71a7123-db02-4e9e-b2fc-b3a2a97edce6/volumes"
Sep 11 18:36:49.914095 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:49.914081    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8430aae-66f8-46bd-a6ea-ed9503951939" path="/var/lib/kubelet/pods/a8430aae-66f8-46bd-a6ea-ed9503951939/volumes"
Sep 11 18:36:50.116769 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:50.116738    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:50.904658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:50.904619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:51.765786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:51.765759    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:52.492582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:52.492548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.168656 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.168592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.173275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.173239    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"]
Sep 11 18:36:53.202934 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.202906    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-553596-29819190-gvsqn"]
Sep 11 18:36:53.253563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.253536    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"]
Sep 11 18:36:53.253799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.253774    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr" podUID="feac80ec-5627-4b6e-8e32-6d9435693ae0" containerName="payload-processing" containerID="cri-o://cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d" gracePeriod=30
Sep 11 18:36:53.334570 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.334536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.435252 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.435226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.506053 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.506031    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:36:53.518368 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.518341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.579464 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.579438    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.581693 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.581674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qdv9\" (UniqueName: \"kubernetes.io/projected/feac80ec-5627-4b6e-8e32-6d9435693ae0-kube-api-access-4qdv9\") pod \"feac80ec-5627-4b6e-8e32-6d9435693ae0\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") "
Sep 11 18:36:53.581778 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.581770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feac80ec-5627-4b6e-8e32-6d9435693ae0-plugins-config-volume\") pod \"feac80ec-5627-4b6e-8e32-6d9435693ae0\" (UID: \"feac80ec-5627-4b6e-8e32-6d9435693ae0\") "
Sep 11 18:36:53.582165 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.582144    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feac80ec-5627-4b6e-8e32-6d9435693ae0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "feac80ec-5627-4b6e-8e32-6d9435693ae0" (UID: "feac80ec-5627-4b6e-8e32-6d9435693ae0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:36:53.584111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.583982    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feac80ec-5627-4b6e-8e32-6d9435693ae0-kube-api-access-4qdv9" (OuterVolumeSpecName: "kube-api-access-4qdv9") pod "feac80ec-5627-4b6e-8e32-6d9435693ae0" (UID: "feac80ec-5627-4b6e-8e32-6d9435693ae0"). InnerVolumeSpecName "kube-api-access-4qdv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:53.626755 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.626723    2563 generic.go:356] "Generic (PLEG): container finished" podID="feac80ec-5627-4b6e-8e32-6d9435693ae0" containerID="cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d" exitCode=0
Sep 11 18:36:53.626911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.626786    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"
Sep 11 18:36:53.626911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.626808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr" event={"ID":"feac80ec-5627-4b6e-8e32-6d9435693ae0","Type":"ContainerDied","Data":"cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d"}
Sep 11 18:36:53.626911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.626876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr" event={"ID":"feac80ec-5627-4b6e-8e32-6d9435693ae0","Type":"ContainerDied","Data":"453d354f9b5e78c7776d53c5594bd3bd65609c83a27a5385b820ab3958b8357e"}
Sep 11 18:36:53.626911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.626907    2563 scope.go:117] "RemoveContainer" containerID="cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d"
Sep 11 18:36:53.636423 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.636402    2563 scope.go:117] "RemoveContainer" containerID="cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d"
Sep 11 18:36:53.636774 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:53.636753    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d\": container with ID starting with cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d not found: ID does not exist" containerID="cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d"
Sep 11 18:36:53.636881 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.636786    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d"} err="failed to get container status \"cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d\": rpc error: code = NotFound desc = could not find container \"cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d\": container with ID starting with cb10c2e0f0b666e301798673533c1fdcfc953ec0e80be3fc32985b55d251eb6d not found: ID does not exist"
Sep 11 18:36:53.653100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.653077    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"]
Sep 11 18:36:53.658960 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.658935    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-553596-7cfc8b6869-lhhmr"]
Sep 11 18:36:53.683422 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.683372    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qdv9\" (UniqueName: \"kubernetes.io/projected/feac80ec-5627-4b6e-8e32-6d9435693ae0-kube-api-access-4qdv9\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:53.683422 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.683393    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/feac80ec-5627-4b6e-8e32-6d9435693ae0-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:53.863374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.863341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:53.913935 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.913900    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4dcb34-e6bd-48da-a121-dd77e4058a97" path="/var/lib/kubelet/pods/7e4dcb34-e6bd-48da-a121-dd77e4058a97/volumes"
Sep 11 18:36:53.914382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.914363    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feac80ec-5627-4b6e-8e32-6d9435693ae0" path="/var/lib/kubelet/pods/feac80ec-5627-4b6e-8e32-6d9435693ae0/volumes"
Sep 11 18:36:53.962379 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:53.962319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:54.700174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:54.700143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:55.420714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:55.420680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:55.639251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:55.639219    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:56.495740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:56.495701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:56.946915 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:56.946887    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:57.052155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.052125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:57.217114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.217023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:57.224570 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.224545    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"]
Sep 11 18:36:57.224806 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.224782    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy" containerID="cri-o://d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e" gracePeriod=30
Sep 11 18:36:57.318222 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.318190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:57.370111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.370072    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv" podUID="21f1a971-6336-4a0b-8cac-2f10b64fec3c" containerName="main" containerID="cri-o://d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa" gracePeriod=45
Sep 11 18:36:57.629781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.629758    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:36:57.656757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.656723    2563 generic.go:356] "Generic (PLEG): container finished" podID="21f1a971-6336-4a0b-8cac-2f10b64fec3c" containerID="d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa" exitCode=0
Sep 11 18:36:57.656969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.656943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv" event={"ID":"21f1a971-6336-4a0b-8cac-2f10b64fec3c","Type":"ContainerDied","Data":"d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa"}
Sep 11 18:36:57.657043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.656995    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv" event={"ID":"21f1a971-6336-4a0b-8cac-2f10b64fec3c","Type":"ContainerDied","Data":"4409a06bfd155da95ba639c45d1c50693f6a229b898708bde53d33668a39b113"}
Sep 11 18:36:57.657043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.657021    2563 scope.go:117] "RemoveContainer" containerID="d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa"
Sep 11 18:36:57.657199 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.657182    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"
Sep 11 18:36:57.667301 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.667239    2563 scope.go:117] "RemoveContainer" containerID="d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa"
Sep 11 18:36:57.667615 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:36:57.667594    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa\": container with ID starting with d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa not found: ID does not exist" containerID="d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa"
Sep 11 18:36:57.667688 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.667629    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa"} err="failed to get container status \"d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa\": rpc error: code = NotFound desc = could not find container \"d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa\": container with ID starting with d76d0c5bdaff21c494ef557c8d01c25b4f97e13f236efd0c1e215f9eb4a333fa not found: ID does not exist"
Sep 11 18:36:57.720852 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.720817    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tls-certs\") pod \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") "
Sep 11 18:36:57.721027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.720963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ft5m\" (UniqueName: \"kubernetes.io/projected/21f1a971-6336-4a0b-8cac-2f10b64fec3c-kube-api-access-5ft5m\") pod \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") "
Sep 11 18:36:57.721027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.720999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-dshm\") pod \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") "
Sep 11 18:36:57.721138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.721055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-home\") pod \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") "
Sep 11 18:36:57.721138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.721084    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tmp-dir\") pod \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") "
Sep 11 18:36:57.721138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.721110    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-model-cache\") pod \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\" (UID: \"21f1a971-6336-4a0b-8cac-2f10b64fec3c\") "
Sep 11 18:36:57.721567 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.721533    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-home" (OuterVolumeSpecName: "home") pod "21f1a971-6336-4a0b-8cac-2f10b64fec3c" (UID: "21f1a971-6336-4a0b-8cac-2f10b64fec3c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:57.722616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.722564    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-model-cache" (OuterVolumeSpecName: "model-cache") pod "21f1a971-6336-4a0b-8cac-2f10b64fec3c" (UID: "21f1a971-6336-4a0b-8cac-2f10b64fec3c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:57.722740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.722681    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "21f1a971-6336-4a0b-8cac-2f10b64fec3c" (UID: "21f1a971-6336-4a0b-8cac-2f10b64fec3c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:57.723473 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.723417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f1a971-6336-4a0b-8cac-2f10b64fec3c-kube-api-access-5ft5m" (OuterVolumeSpecName: "kube-api-access-5ft5m") pod "21f1a971-6336-4a0b-8cac-2f10b64fec3c" (UID: "21f1a971-6336-4a0b-8cac-2f10b64fec3c"). InnerVolumeSpecName "kube-api-access-5ft5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:36:57.723473 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.723660    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "21f1a971-6336-4a0b-8cac-2f10b64fec3c" (UID: "21f1a971-6336-4a0b-8cac-2f10b64fec3c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:36:57.724047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.724018    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-dshm" (OuterVolumeSpecName: "dshm") pod "21f1a971-6336-4a0b-8cac-2f10b64fec3c" (UID: "21f1a971-6336-4a0b-8cac-2f10b64fec3c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:36:57.822750 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.822723    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:57.822750 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.822747    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:57.822938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.822779    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:57.822938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.822794    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21f1a971-6336-4a0b-8cac-2f10b64fec3c-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:57.822938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.822806    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21f1a971-6336-4a0b-8cac-2f10b64fec3c-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:57.822938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.822827    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ft5m\" (UniqueName: \"kubernetes.io/projected/21f1a971-6336-4a0b-8cac-2f10b64fec3c-kube-api-access-5ft5m\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:36:57.977150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.977116    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"]
Sep 11 18:36:57.980657 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:57.980633    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-553596/distinct-w4-553596-kserve-76996b7866-22khv"]
Sep 11 18:36:58.099809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:58.099730    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:58.954151 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:58.954117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:59.021012 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:59.020982    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:59.882282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:59.882232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:36:59.913335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:36:59.913304    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f1a971-6336-4a0b-8cac-2f10b64fec3c" path="/var/lib/kubelet/pods/21f1a971-6336-4a0b-8cac-2f10b64fec3c/volumes"
Sep 11 18:37:00.534426 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:00.534393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:01.165123 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:01.165090    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:01.223073 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:01.223041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:01.289106 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:01.283041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:01.431331 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:01.431226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:02.481602 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.481580    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:37:02.572937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.572905    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-token\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.572974    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c2tg\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-kube-api-access-2c2tg\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-envoy\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-data\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-podinfo\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573119    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istiod-ca-cert\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573165    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-socket\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-credential-socket\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573253    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-certs\") pod \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\" (UID: \"7b49cfa6-71cc-4f8a-9ac2-744518df92c4\") "
Sep 11 18:37:02.573523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573432    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:02.573644 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573616    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:02.573704 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:02.573797 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573778    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.573867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573802    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.573867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573815    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.573867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:02.573867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.573819    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:02.575307 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.575255    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:37:02.575393 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.575287    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-kube-api-access-2c2tg" (OuterVolumeSpecName: "kube-api-access-2c2tg") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "kube-api-access-2c2tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:02.575550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.575522    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:37:02.575550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.575541    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b49cfa6-71cc-4f8a-9ac2-744518df92c4" (UID: "7b49cfa6-71cc-4f8a-9ac2-744518df92c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:02.675118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.675094    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.675118 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.675114    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c2tg\" (UniqueName: \"kubernetes.io/projected/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-kube-api-access-2c2tg\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.675251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.675124    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.675251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.675134    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.675251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.675142    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.675251 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.675151    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b49cfa6-71cc-4f8a-9ac2-744518df92c4-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:02.687672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.687643    2563 generic.go:356] "Generic (PLEG): container finished" podID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerID="d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e" exitCode=0
Sep 11 18:37:02.687757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.687685    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"
Sep 11 18:37:02.687757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.687686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" event={"ID":"7b49cfa6-71cc-4f8a-9ac2-744518df92c4","Type":"ContainerDied","Data":"d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e"}
Sep 11 18:37:02.687757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.687740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z" event={"ID":"7b49cfa6-71cc-4f8a-9ac2-744518df92c4","Type":"ContainerDied","Data":"217315a1dc23edb40f578c7749eed55ef97e8468a4744cfc891bec82225730c0"}
Sep 11 18:37:02.687865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.687763    2563 scope.go:117] "RemoveContainer" containerID="d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e"
Sep 11 18:37:02.696711 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.696693    2563 scope.go:117] "RemoveContainer" containerID="3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"
Sep 11 18:37:02.704557 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.704540    2563 scope.go:117] "RemoveContainer" containerID="d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e"
Sep 11 18:37:02.704809 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:37:02.704791    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e\": container with ID starting with d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e not found: ID does not exist" containerID="d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e"
Sep 11 18:37:02.704863 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.704816    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e"} err="failed to get container status \"d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e\": rpc error: code = NotFound desc = could not find container \"d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e\": container with ID starting with d2e42d68917232e90e9982cc5ba864d3d443e9689409daf215ad33c991f8e88e not found: ID does not exist"
Sep 11 18:37:02.704863 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.704832    2563 scope.go:117] "RemoveContainer" containerID="3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"
Sep 11 18:37:02.705071 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:37:02.705056    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995\": container with ID starting with 3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995 not found: ID does not exist" containerID="3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"
Sep 11 18:37:02.705116 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.705076    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995"} err="failed to get container status \"3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995\": rpc error: code = NotFound desc = could not find container \"3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995\": container with ID starting with 3ef06dc69a27cad06cd8c927057ab5c716d52f285778e3f340c404adb5936995 not found: ID does not exist"
Sep 11 18:37:02.711593 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.711572    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"]
Sep 11 18:37:02.715074 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:02.715055    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b77bc8-openshift-default-67d7bc654d-phz7z"]
Sep 11 18:37:03.913274 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:03.913231    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" path="/var/lib/kubelet/pods/7b49cfa6-71cc-4f8a-9ac2-744518df92c4/volumes"
Sep 11 18:37:06.930959 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:06.930930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:06.988417 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:06.988389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:07.399041 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:07.399008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:07.757584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:07.757497    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:08.328644 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:08.328601    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:08.427692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:08.427645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:08.495866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:08.495829    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:08.560971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:08.560941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:08.885887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:08.885846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:08.986455 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:08.986425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:09.626517 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:09.626476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:10.315170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:10.315137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:11.191703 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:11.191668    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:14.960285 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:14.960230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:15.026939 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:15.026911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:15.243958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:15.243877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:15.456286 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:15.456245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:15.703884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:15.703851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:16.566253 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:16.566220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:17.394658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:17.394619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:18.185006 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:18.184974    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:18.857554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:18.857524    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:19.089061 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:19.089035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:22.596062 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:22.596026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:26.262466 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.262385    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:26.297305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.297272    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"]
Sep 11 18:37:26.297550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.297525    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56" podUID="39f77b50-fdee-44b3-b715-a291cf34720f" containerName="payload-pre-processing" containerID="cri-o://f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b" gracePeriod=30
Sep 11 18:37:26.324312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.324289    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:26.429677 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.429641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:26.499953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.499919    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:26.551288 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.551236    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:37:26.688564 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.688533    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f77b50-fdee-44b3-b715-a291cf34720f-plugins-config-volume\") pod \"39f77b50-fdee-44b3-b715-a291cf34720f\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") "
Sep 11 18:37:26.688746 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.688688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsq95\" (UniqueName: \"kubernetes.io/projected/39f77b50-fdee-44b3-b715-a291cf34720f-kube-api-access-rsq95\") pod \"39f77b50-fdee-44b3-b715-a291cf34720f\" (UID: \"39f77b50-fdee-44b3-b715-a291cf34720f\") "
Sep 11 18:37:26.688975 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.688947    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39f77b50-fdee-44b3-b715-a291cf34720f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39f77b50-fdee-44b3-b715-a291cf34720f" (UID: "39f77b50-fdee-44b3-b715-a291cf34720f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:37:26.690788 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.690757    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f77b50-fdee-44b3-b715-a291cf34720f-kube-api-access-rsq95" (OuterVolumeSpecName: "kube-api-access-rsq95") pod "39f77b50-fdee-44b3-b715-a291cf34720f" (UID: "39f77b50-fdee-44b3-b715-a291cf34720f"). InnerVolumeSpecName "kube-api-access-rsq95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:37:26.789591 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.789527    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsq95\" (UniqueName: \"kubernetes.io/projected/39f77b50-fdee-44b3-b715-a291cf34720f-kube-api-access-rsq95\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:26.789591 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.789550    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39f77b50-fdee-44b3-b715-a291cf34720f-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:37:26.818141 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.818113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:26.828925 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.828898    2563 generic.go:356] "Generic (PLEG): container finished" podID="39f77b50-fdee-44b3-b715-a291cf34720f" containerID="f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b" exitCode=0
Sep 11 18:37:26.829027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.828941    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"
Sep 11 18:37:26.829027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.828982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56" event={"ID":"39f77b50-fdee-44b3-b715-a291cf34720f","Type":"ContainerDied","Data":"f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b"}
Sep 11 18:37:26.829114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.829040    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56" event={"ID":"39f77b50-fdee-44b3-b715-a291cf34720f","Type":"ContainerDied","Data":"2af183f8ea11dbde77bbe7f4c23b236fb66d50c6f1045491cd3296933b62e4d9"}
Sep 11 18:37:26.829114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.829068    2563 scope.go:117] "RemoveContainer" containerID="f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b"
Sep 11 18:37:26.838948 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.838926    2563 scope.go:117] "RemoveContainer" containerID="f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b"
Sep 11 18:37:26.839213 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:37:26.839195    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b\": container with ID starting with f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b not found: ID does not exist" containerID="f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b"
Sep 11 18:37:26.839330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.839222    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b"} err="failed to get container status \"f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b\": rpc error: code = NotFound desc = could not find container \"f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b\": container with ID starting with f68b01345ff318c479ff859b1485ee85517b5c5b08f72e239d6abec480372d6b not found: ID does not exist"
Sep 11 18:37:26.850505 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.850478    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"]
Sep 11 18:37:26.852435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.852415    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6c3180-567978b4c4-q5v56"]
Sep 11 18:37:26.919116 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:26.919089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:27.507665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:27.507625    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:27.912490 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:27.912458    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39f77b50-fdee-44b3-b715-a291cf34720f" path="/var/lib/kubelet/pods/39f77b50-fdee-44b3-b715-a291cf34720f/volumes"
Sep 11 18:37:28.184303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:28.184207    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:28.991152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:28.991121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:32.131298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:32.131253    2563 scope.go:117] "RemoveContainer" containerID="521deb98263959df9204ccc7351391ca90e0baf89747c7b6636b7804210c6414"
Sep 11 18:37:33.100820 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:33.100786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:33.200953 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:33.200927    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:37:33.368616 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:37:33.368533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:38:06.900078 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900044    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"]
Sep 11 18:38:06.900708 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900688    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900711    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900722    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a71a7123-db02-4e9e-b2fc-b3a2a97edce6" containerName="payload-processing"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900731    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a71a7123-db02-4e9e-b2fc-b3a2a97edce6" containerName="payload-processing"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900745    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e8bd2f-899c-4378-8172-4d5f6a0e3634" containerName="payload-pre-processing"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900754    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e8bd2f-899c-4378-8172-4d5f6a0e3634" containerName="payload-pre-processing"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900765    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748e1efb-e270-4e97-9006-b59f9aab0dee" containerName="payload-pre-processing"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900774    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="748e1efb-e270-4e97-9006-b59f9aab0dee" containerName="payload-pre-processing"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900784    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8430aae-66f8-46bd-a6ea-ed9503951939" containerName="maas-api"
Sep 11 18:38:06.900796 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900792    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8430aae-66f8-46bd-a6ea-ed9503951939" containerName="maas-api"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900802    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900811    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900823    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900831    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900849    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feac80ec-5627-4b6e-8e32-6d9435693ae0" containerName="payload-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900857    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="feac80ec-5627-4b6e-8e32-6d9435693ae0" containerName="payload-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900875    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f1a971-6336-4a0b-8cac-2f10b64fec3c" containerName="main"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900884    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f1a971-6336-4a0b-8cac-2f10b64fec3c" containerName="main"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900910    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900918    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900926    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f77b50-fdee-44b3-b715-a291cf34720f" containerName="payload-pre-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.900935    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f77b50-fdee-44b3-b715-a291cf34720f" containerName="payload-pre-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901055    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a71a7123-db02-4e9e-b2fc-b3a2a97edce6" containerName="payload-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901069    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="748e1efb-e270-4e97-9006-b59f9aab0dee" containerName="payload-pre-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901080    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901089    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="feac80ec-5627-4b6e-8e32-6d9435693ae0" containerName="payload-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901099    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8430aae-66f8-46bd-a6ea-ed9503951939" containerName="maas-api"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901107    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b49cfa6-71cc-4f8a-9ac2-744518df92c4" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901117    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901126    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e8bd2f-899c-4378-8172-4d5f6a0e3634" containerName="payload-pre-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901135    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2d2943-36ae-43df-b7e4-bf5fb239049c" containerName="istio-proxy"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901145    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f77b50-fdee-44b3-b715-a291cf34720f" containerName="payload-pre-processing"
Sep 11 18:38:06.901281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.901154    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f1a971-6336-4a0b-8cac-2f10b64fec3c" containerName="main"
Sep 11 18:38:06.904550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.904531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:06.906741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.906719    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5faf38\"/\"kube-root-ca.crt\""
Sep 11 18:38:06.907580 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.907546    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5faf38\"/\"default-dockercfg-qwk46\""
Sep 11 18:38:06.907683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.907618    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5faf38\"/\"sub-test-model-5faf38-kserve-self-signed-certs\""
Sep 11 18:38:06.907827 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.907811    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5faf38\"/\"openshift-service-ca.crt\""
Sep 11 18:38:06.913283 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.913238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"]
Sep 11 18:38:06.927401 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:06.927378    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:38:07.041799 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.041778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-dshm\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.041897 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.041804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrn2m\" (UniqueName: \"kubernetes.io/projected/233422f1-4fa2-4e99-8f43-f86b412241e1-kube-api-access-rrn2m\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.041897 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.041842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/233422f1-4fa2-4e99-8f43-f86b412241e1-tls-certs\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.041975 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.041914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-tmp-dir\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.041975 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.041961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-home\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.042110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.042091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-model-cache\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.142829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.142797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-tmp-dir\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.142992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.142838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-home\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.142992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.142898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-model-cache\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.142992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.142951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-dshm\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.142992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.142975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrn2m\" (UniqueName: \"kubernetes.io/projected/233422f1-4fa2-4e99-8f43-f86b412241e1-kube-api-access-rrn2m\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.143284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.143015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/233422f1-4fa2-4e99-8f43-f86b412241e1-tls-certs\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.143284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.143238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-tmp-dir\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.143433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.143294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-home\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.143433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.143382    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-model-cache\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.145405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.145384    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-dshm\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.145535 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.145515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/233422f1-4fa2-4e99-8f43-f86b412241e1-tls-certs\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.150615 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.150558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrn2m\" (UniqueName: \"kubernetes.io/projected/233422f1-4fa2-4e99-8f43-f86b412241e1-kube-api-access-rrn2m\") pod \"sub-test-model-5faf38-kserve-76fcfb4797-h5qgw\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") " pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.215609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.215584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:07.495091 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:07.495065    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"]
Sep 11 18:38:07.496497 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:38:07.496468    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod233422f1_4fa2_4e99_8f43_f86b412241e1.slice/crio-1785d143f36dce0550101268d0e5e3c7e9f105e66fc00f16f4d8f344e804fe93 WatchSource:0}: Error finding container 1785d143f36dce0550101268d0e5e3c7e9f105e66fc00f16f4d8f344e804fe93: Status 404 returned error can't find the container with id 1785d143f36dce0550101268d0e5e3c7e9f105e66fc00f16f4d8f344e804fe93
Sep 11 18:38:08.039981 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:08.039945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw" event={"ID":"233422f1-4fa2-4e99-8f43-f86b412241e1","Type":"ContainerStarted","Data":"4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2"}
Sep 11 18:38:08.039981 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:08.039984    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw" event={"ID":"233422f1-4fa2-4e99-8f43-f86b412241e1","Type":"ContainerStarted","Data":"1785d143f36dce0550101268d0e5e3c7e9f105e66fc00f16f4d8f344e804fe93"}
Sep 11 18:38:08.056770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:08.056729    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw" podStartSLOduration=2.05671443 podStartE2EDuration="2.05671443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:38:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:38:08.05542703 +0000 UTC m=+1656.648627587" watchObservedRunningTime="2026-09-11 18:38:08.05671443 +0000 UTC m=+1656.649914988"
Sep 11 18:38:15.193047 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:15.193015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:38:15.302592 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:15.302566    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:38:17.216324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:17.216297    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:17.216709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:17.216543    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:17.229051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:17.229028    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:18.102034 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:18.102004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:38:26.701861 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:26.701824    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:38:50.506756 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:50.506699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:38:51.605390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:51.605354    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"]
Sep 11 18:38:51.623546 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:38:51.623513    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:39:03.033125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:03.033054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:39:03.134544 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:03.134517    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:39:06.624950 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.624911    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw" podUID="233422f1-4fa2-4e99-8f43-f86b412241e1" containerName="main" containerID="cri-o://4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2" gracePeriod=45
Sep 11 18:39:06.867642 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.867616    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:39:06.968828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.968757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-tmp-dir\") pod \"233422f1-4fa2-4e99-8f43-f86b412241e1\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") "
Sep 11 18:39:06.968828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.968795    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/233422f1-4fa2-4e99-8f43-f86b412241e1-tls-certs\") pod \"233422f1-4fa2-4e99-8f43-f86b412241e1\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") "
Sep 11 18:39:06.968828 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.968814    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-home\") pod \"233422f1-4fa2-4e99-8f43-f86b412241e1\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") "
Sep 11 18:39:06.969026 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.968839    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-model-cache\") pod \"233422f1-4fa2-4e99-8f43-f86b412241e1\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") "
Sep 11 18:39:06.969026 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.968882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrn2m\" (UniqueName: \"kubernetes.io/projected/233422f1-4fa2-4e99-8f43-f86b412241e1-kube-api-access-rrn2m\") pod \"233422f1-4fa2-4e99-8f43-f86b412241e1\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") "
Sep 11 18:39:06.969026 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.968909    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-dshm\") pod \"233422f1-4fa2-4e99-8f43-f86b412241e1\" (UID: \"233422f1-4fa2-4e99-8f43-f86b412241e1\") "
Sep 11 18:39:06.969173 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.969083    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "233422f1-4fa2-4e99-8f43-f86b412241e1" (UID: "233422f1-4fa2-4e99-8f43-f86b412241e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:06.969173 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.969106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-home" (OuterVolumeSpecName: "home") pod "233422f1-4fa2-4e99-8f43-f86b412241e1" (UID: "233422f1-4fa2-4e99-8f43-f86b412241e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:06.969312 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.969180    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "233422f1-4fa2-4e99-8f43-f86b412241e1" (UID: "233422f1-4fa2-4e99-8f43-f86b412241e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:06.969497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.969477    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:39:06.969554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.969503    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:39:06.969554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.969520    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:39:06.971027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.971004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-dshm" (OuterVolumeSpecName: "dshm") pod "233422f1-4fa2-4e99-8f43-f86b412241e1" (UID: "233422f1-4fa2-4e99-8f43-f86b412241e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:39:06.971107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.971087    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233422f1-4fa2-4e99-8f43-f86b412241e1-kube-api-access-rrn2m" (OuterVolumeSpecName: "kube-api-access-rrn2m") pod "233422f1-4fa2-4e99-8f43-f86b412241e1" (UID: "233422f1-4fa2-4e99-8f43-f86b412241e1"). InnerVolumeSpecName "kube-api-access-rrn2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:39:06.971171 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:06.971152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/233422f1-4fa2-4e99-8f43-f86b412241e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "233422f1-4fa2-4e99-8f43-f86b412241e1" (UID: "233422f1-4fa2-4e99-8f43-f86b412241e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:39:07.070500 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.070477    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrn2m\" (UniqueName: \"kubernetes.io/projected/233422f1-4fa2-4e99-8f43-f86b412241e1-kube-api-access-rrn2m\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:39:07.070500 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.070497    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/233422f1-4fa2-4e99-8f43-f86b412241e1-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:39:07.070629 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.070506    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/233422f1-4fa2-4e99-8f43-f86b412241e1-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:39:07.342977 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.342947    2563 generic.go:356] "Generic (PLEG): container finished" podID="233422f1-4fa2-4e99-8f43-f86b412241e1" containerID="4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2" exitCode=0
Sep 11 18:39:07.343138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.342987    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"
Sep 11 18:39:07.343138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.343038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw" event={"ID":"233422f1-4fa2-4e99-8f43-f86b412241e1","Type":"ContainerDied","Data":"4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2"}
Sep 11 18:39:07.343138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.343088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw" event={"ID":"233422f1-4fa2-4e99-8f43-f86b412241e1","Type":"ContainerDied","Data":"1785d143f36dce0550101268d0e5e3c7e9f105e66fc00f16f4d8f344e804fe93"}
Sep 11 18:39:07.343138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.343109    2563 scope.go:117] "RemoveContainer" containerID="4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2"
Sep 11 18:39:07.352079 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.352062    2563 scope.go:117] "RemoveContainer" containerID="4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2"
Sep 11 18:39:07.352344 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:39:07.352324    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2\": container with ID starting with 4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2 not found: ID does not exist" containerID="4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2"
Sep 11 18:39:07.352443 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.352350    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2"} err="failed to get container status \"4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2\": rpc error: code = NotFound desc = could not find container \"4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2\": container with ID starting with 4470516c8ce052394aa0973361121381bda384ab7470c36eed9b21028fa22de2 not found: ID does not exist"
Sep 11 18:39:07.364973 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.364951    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"]
Sep 11 18:39:07.367758 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.367739    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5faf38/sub-test-model-5faf38-kserve-76fcfb4797-h5qgw"]
Sep 11 18:39:07.912727 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:07.912698    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="233422f1-4fa2-4e99-8f43-f86b412241e1" path="/var/lib/kubelet/pods/233422f1-4fa2-4e99-8f43-f86b412241e1/volumes"
Sep 11 18:39:25.109883 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:25.109835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:39:25.959219 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:25.959186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:39:26.017710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:26.017679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:39:29.264097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:39:29.264064    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:02.835847 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:02.835813    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:32.017744 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:32.017715    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:40:32.022624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:32.022604    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:40:33.751437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:33.751406    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:48.058005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:48.057952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:53.508392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:53.508360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:54.495339 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:54.495305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:54.869647 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:54.869615    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:55.132800 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.132724    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"]
Sep 11 18:40:55.133372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.133342    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="233422f1-4fa2-4e99-8f43-f86b412241e1" containerName="main"
Sep 11 18:40:55.133372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.133370    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="233422f1-4fa2-4e99-8f43-f86b412241e1" containerName="main"
Sep 11 18:40:55.133609 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.133490    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="233422f1-4fa2-4e99-8f43-f86b412241e1" containerName="main"
Sep 11 18:40:55.136715 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.136693    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.138849 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.138831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-765dd1-openshift-default-dockercfg-qdwlm\""
Sep 11 18:40:55.147869 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.147849    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"]
Sep 11 18:40:55.158730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.158708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:55.204540 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204517    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brjb8\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-kube-api-access-brjb8\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-credential-socket\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istiod-ca-cert\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204628    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-envoy\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204768 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-token\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204768 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-certs\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204768 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-podinfo\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204862 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-socket\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.204862 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.204806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-data\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.305994 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.305960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-data\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306162 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brjb8\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-kube-api-access-brjb8\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306162 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-credential-socket\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istiod-ca-cert\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-envoy\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-token\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306471 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-data\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306471 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-certs\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-podinfo\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306575 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-socket\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306729 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306711    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-certs\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306849 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-credential-socket\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.306933 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.306911    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-socket\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.307139 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.307107    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istiod-ca-cert\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.308521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.308495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-envoy\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.308905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.308884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-podinfo\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.314381 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.314360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-token\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.314780 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.314738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brjb8\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-kube-api-access-brjb8\") pod \"e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") " pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.448124 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.448065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:55.728223 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.728194    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"]
Sep 11 18:40:55.729934 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:40:55.729909    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc8e855_11a4_4ffc_b116_6b06aa1cfd40.slice/crio-ad9ed6b6cfd2075f556ef75d4b1f84ec7bf8e0e212d73a76edc59f972dd7892e WatchSource:0}: Error finding container ad9ed6b6cfd2075f556ef75d4b1f84ec7bf8e0e212d73a76edc59f972dd7892e: Status 404 returned error can't find the container with id ad9ed6b6cfd2075f556ef75d4b1f84ec7bf8e0e212d73a76edc59f972dd7892e
Sep 11 18:40:55.732582 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.732540    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:40:55.732660 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.732634    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:40:55.732699 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.732678    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:40:55.831889 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.831865    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:55.901537 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.901504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" event={"ID":"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40","Type":"ContainerStarted","Data":"1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"}
Sep 11 18:40:55.901966 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.901568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" event={"ID":"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40","Type":"ContainerStarted","Data":"ad9ed6b6cfd2075f556ef75d4b1f84ec7bf8e0e212d73a76edc59f972dd7892e"}
Sep 11 18:40:55.923993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:55.923936    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podStartSLOduration=0.923917282 podStartE2EDuration="923.917282ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:40:55.919859659 +0000 UTC m=+1824.513060215" watchObservedRunningTime="2026-09-11 18:40:55.923917282 +0000 UTC m=+1824.517117842"
Sep 11 18:40:56.449135 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:56.449099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:56.453843 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:56.453816    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:56.909046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:56.909007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:56.909987 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:56.909966    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:40:57.091367 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:57.091330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:57.887516 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:57.887477    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:57.988037 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:57.988003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:58.721006 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:58.720973    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:59.658997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:59.658970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:59.758654 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:59.758624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:40:59.857254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:40:59.857228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:00.111377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.111331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:00.206024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.205993    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"]
Sep 11 18:41:00.208163 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.208137    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:00.209737 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.209720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.212125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.212108    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-765dd1\""
Sep 11 18:41:00.223573 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.223551    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"]
Sep 11 18:41:00.251187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.251159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srf5r\" (UniqueName: \"kubernetes.io/projected/e6857185-862b-401b-af61-7cc744160028-kube-api-access-srf5r\") pod \"payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.251398 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.251378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6857185-862b-401b-af61-7cc744160028-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.352993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.352946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6857185-862b-401b-af61-7cc744160028-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.353181 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.353003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srf5r\" (UniqueName: \"kubernetes.io/projected/e6857185-862b-401b-af61-7cc744160028-kube-api-access-srf5r\") pod \"payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.353954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.353917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6857185-862b-401b-af61-7cc744160028-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.362240 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.362179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srf5r\" (UniqueName: \"kubernetes.io/projected/e6857185-862b-401b-af61-7cc744160028-kube-api-access-srf5r\") pod \"payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.520945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.520916    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.795445 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.795413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"]
Sep 11 18:41:00.796751 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:41:00.796686    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6857185_862b_401b_af61_7cc744160028.slice/crio-e38d241e3b1d45d57b3990cb52df1be2962cbdaaf811e9eb13377718c2ffba19 WatchSource:0}: Error finding container e38d241e3b1d45d57b3990cb52df1be2962cbdaaf811e9eb13377718c2ffba19: Status 404 returned error can't find the container with id e38d241e3b1d45d57b3990cb52df1be2962cbdaaf811e9eb13377718c2ffba19
Sep 11 18:41:00.931798 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.931765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp" event={"ID":"e6857185-862b-401b-af61-7cc744160028","Type":"ContainerStarted","Data":"7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929"}
Sep 11 18:41:00.931958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.931813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp" event={"ID":"e6857185-862b-401b-af61-7cc744160028","Type":"ContainerStarted","Data":"e38d241e3b1d45d57b3990cb52df1be2962cbdaaf811e9eb13377718c2ffba19"}
Sep 11 18:41:00.931958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.931906    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:00.949982 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:00.949941    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp" podStartSLOduration=0.949927845 podStartE2EDuration="949.927845ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:41:00.947200794 +0000 UTC m=+1829.540401352" watchObservedRunningTime="2026-09-11 18:41:00.949927845 +0000 UTC m=+1829.543128436"
Sep 11 18:41:11.938669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:11.938640    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:41:12.950409 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:12.950367    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 18:41:12.950826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:12.950430    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 18:41:14.000922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:14.000824    2563 generic.go:356] "Generic (PLEG): container finished" podID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerID="1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606" exitCode=0
Sep 11 18:41:14.000922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:14.000903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" event={"ID":"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40","Type":"ContainerDied","Data":"1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"}
Sep 11 18:41:14.001456 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:14.001223    2563 scope.go:117] "RemoveContainer" containerID="1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"
Sep 11 18:41:14.002245 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:14.002213    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:41:14.002370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:14.002303    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:41:14.002370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:14.002344    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:41:15.018719 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:15.018670    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" event={"ID":"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40","Type":"ContainerStarted","Data":"e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47"}
Sep 11 18:41:15.448891 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:15.448852    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:41:15.449082 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:15.449059    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 11 18:41:15.449136 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:15.449107    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 11 18:41:16.449313 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:16.449280    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 11 18:41:16.449676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:16.449334    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 11 18:41:16.986236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:16.986185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:17.448739 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:17.448698    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 11 18:41:17.448918 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:17.448764    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 11 18:41:18.448521 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:18.448486    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 11 18:41:18.448887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:18.448548    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 11 18:41:19.448862 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:19.448827    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 11 18:41:19.449227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:19.448883    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 11 18:41:20.458707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:20.458672    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:41:20.459131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:20.459032    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:41:20.459735 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:20.459717    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:41:21.517716 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:21.517684    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:22.761629 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:22.761592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:22.862275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:22.862211    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:23.433282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:23.433227    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:23.591476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:23.591442    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:23.734008 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:23.733927    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:24.323152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:24.323121    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:24.423786 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:24.423756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:25.025673 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:25.025639    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:26.296528 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:26.296471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:27.092580 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:27.092534    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:27.181051 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:27.181020    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:28.006698 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:28.006665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:28.895563 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:28.895531    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:29.583379 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:29.583345    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:29.684252 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:29.684222    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:29.783781 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:29.783750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:30.051692 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:30.051650    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:30.136802 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:30.136778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:32.208275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:32.208230    2563 scope.go:117] "RemoveContainer" containerID="7fc2f5b5d3d2582eebb86c83ff43df4c05eefe9b15ebea0f359c61ad1dba7ceb"
Sep 11 18:41:51.689465 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:51.689433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:52.083714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:52.083669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:52.500404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:52.500311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:53.181664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:53.181629    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:53.836303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:53.836250    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:54.590895 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:54.590860    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:41:56.998164 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:41:56.998134    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-gc5jg_d1aeb3a3-97e1-4310-a598-0346c7e1231b/payload-processing/0.log"
Sep 11 18:42:08.197652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.197618    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"]
Sep 11 18:42:08.200134 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.200111    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.202592 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.202570    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-765dd1\"/\"openshift-service-ca.crt\""
Sep 11 18:42:08.203460 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.203399    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-765dd1\"/\"kube-root-ca.crt\""
Sep 11 18:42:08.203683 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.203656    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-765dd1\"/\"default-dockercfg-87lqr\""
Sep 11 18:42:08.203794 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.203657    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-765dd1\"/\"ipp-route-765dd1-kserve-self-signed-certs\""
Sep 11 18:42:08.211450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.211426    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"]
Sep 11 18:42:08.237599 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.237577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:08.274100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.274079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d296c813-1fb4-44c4-ad1e-812d14eefd96-tls-certs\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.274200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.274135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-model-cache\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.274200 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.274178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc49q\" (UniqueName: \"kubernetes.io/projected/d296c813-1fb4-44c4-ad1e-812d14eefd96-kube-api-access-wc49q\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.274284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.274202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-home\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.274325 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.274294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-tmp-dir\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.274361 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.274328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-dshm\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375105 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-model-cache\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375222 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc49q\" (UniqueName: \"kubernetes.io/projected/d296c813-1fb4-44c4-ad1e-812d14eefd96-kube-api-access-wc49q\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375222 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-home\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375222 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-tmp-dir\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375222 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-dshm\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d296c813-1fb4-44c4-ad1e-812d14eefd96-tls-certs\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375572 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-model-cache\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375626 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-home\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.375664 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.375621    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-tmp-dir\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.377354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.377336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-dshm\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.377656 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.377637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d296c813-1fb4-44c4-ad1e-812d14eefd96-tls-certs\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.382668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.382646    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc49q\" (UniqueName: \"kubernetes.io/projected/d296c813-1fb4-44c4-ad1e-812d14eefd96-kube-api-access-wc49q\") pod \"ipp-route-765dd1-kserve-78757948c9-p7kfw\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") " pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.513233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.513152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:08.785404 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:08.785380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"]
Sep 11 18:42:08.787697 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:42:08.787668    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd296c813_1fb4_44c4_ad1e_812d14eefd96.slice/crio-e949b7f127600e1004c00c34389fc13acf0d7b1007cf14ffe6e04cb28581f23f WatchSource:0}: Error finding container e949b7f127600e1004c00c34389fc13acf0d7b1007cf14ffe6e04cb28581f23f: Status 404 returned error can't find the container with id e949b7f127600e1004c00c34389fc13acf0d7b1007cf14ffe6e04cb28581f23f
Sep 11 18:42:09.296389 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:09.296353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw" event={"ID":"d296c813-1fb4-44c4-ad1e-812d14eefd96","Type":"ContainerStarted","Data":"4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616"}
Sep 11 18:42:09.296770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:09.296399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw" event={"ID":"d296c813-1fb4-44c4-ad1e-812d14eefd96","Type":"ContainerStarted","Data":"e949b7f127600e1004c00c34389fc13acf0d7b1007cf14ffe6e04cb28581f23f"}
Sep 11 18:42:09.315667 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:09.315629    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw" podStartSLOduration=1.3156161659999999 podStartE2EDuration="1.315616166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:42:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:42:09.313002408 +0000 UTC m=+1897.906202965" watchObservedRunningTime="2026-09-11 18:42:09.315616166 +0000 UTC m=+1897.908816722"
Sep 11 18:42:18.513899 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:18.513866    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:18.514299 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:18.513911    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:18.526230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:18.526207    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:19.361560 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:19.361531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:42:38.133707 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.133677    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-gc5jg_d1aeb3a3-97e1-4310-a598-0346c7e1231b/payload-processing/0.log"
Sep 11 18:42:38.501601 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.501526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:38.776207 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.776170    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"]
Sep 11 18:42:38.784560 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.784528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.787728 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.787698    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-cd2146-openshift-default-dockercfg-jkk67\""
Sep 11 18:42:38.793827 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.793804    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"]
Sep 11 18:42:38.804643 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.804616    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:38.942691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-envoy\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.942691 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942697    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.942896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-credential-socket\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.942896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942770    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-certs\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.942896 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-socket\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.943002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-data\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.943002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.942971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtwb\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-kube-api-access-tjtwb\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.943068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.943025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-token\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:38.943068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:38.943053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-podinfo\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.006583 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.006556    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:39.044088 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-certs\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044088 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-socket\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-data\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtwb\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-kube-api-access-tjtwb\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-token\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-podinfo\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044512 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-socket\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044574 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-certs\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044574 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-envoy\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-data\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044591    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-credential-socket\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.044933 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.044912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-credential-socket\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.045122 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.045104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.046682 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.046662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-envoy\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.047038 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.047016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-podinfo\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.053995 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.053971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtwb\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-kube-api-access-tjtwb\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.055749 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.055730    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-token\") pod \"e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") " pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.099602 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.099578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:39.378022 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.377995    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"]
Sep 11 18:42:39.378984 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:42:39.378952    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbe68d8_9410_46c4_8df3_2055548e6d0b.slice/crio-7dc1a40e0aefbcad20e7a33ff013aea7211aee082c6269bee454c7506d386662 WatchSource:0}: Error finding container 7dc1a40e0aefbcad20e7a33ff013aea7211aee082c6269bee454c7506d386662: Status 404 returned error can't find the container with id 7dc1a40e0aefbcad20e7a33ff013aea7211aee082c6269bee454c7506d386662
Sep 11 18:42:39.381344 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.381308    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:42:39.381427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.381377    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:42:39.381427 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.381403    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:42:39.459281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.459230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" event={"ID":"cfbe68d8-9410-46c4-8df3-2055548e6d0b","Type":"ContainerStarted","Data":"56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"}
Sep 11 18:42:39.459428 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.459307    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" event={"ID":"cfbe68d8-9410-46c4-8df3-2055548e6d0b","Type":"ContainerStarted","Data":"7dc1a40e0aefbcad20e7a33ff013aea7211aee082c6269bee454c7506d386662"}
Sep 11 18:42:39.486714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:39.486667    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podStartSLOduration=1.48665044 podStartE2EDuration="1.48665044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:42:39.485245313 +0000 UTC m=+1928.078445871" watchObservedRunningTime="2026-09-11 18:42:39.48665044 +0000 UTC m=+1928.079850996"
Sep 11 18:42:40.099870 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:40.099841    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:40.105641 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:40.105616    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:40.466805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:40.466726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:40.467725 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:40.467699    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:42:40.689102 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:40.689069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:41.368969 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:41.368916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:41.579199 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:41.579170    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:41.674396 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:41.674301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:42.496442 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:42.496411    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:43.399777 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:43.399741    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:44.056501 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:44.056466    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:44.761377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:44.761340    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:44.861554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:44.861516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:44.963216 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:44.963186    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:45.320300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.320271    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:45.422231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.422198    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:42:45.445612 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.445586    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"]
Sep 11 18:42:45.449429 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.449410    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.452159 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.452134    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-cd2146\""
Sep 11 18:42:45.452551 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.452528    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-cd2146-dockercfg-9x4vb\""
Sep 11 18:42:45.458721 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.458694    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"]
Sep 11 18:42:45.613913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.613839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.613913 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.613884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8rd8\" (UniqueName: \"kubernetes.io/projected/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-kube-api-access-j8rd8\") pod \"payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.714598 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.714565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8rd8\" (UniqueName: \"kubernetes.io/projected/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-kube-api-access-j8rd8\") pod \"payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.714743 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.714683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.715341 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.715318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.722067 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.722043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8rd8\" (UniqueName: \"kubernetes.io/projected/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-kube-api-access-j8rd8\") pod \"payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:45.764346 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:45.764322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:46.036792 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:46.036766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"]
Sep 11 18:42:46.038327 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:42:46.038298    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d099b4_cf86_4998_b2bc_f243bc6e8ada.slice/crio-6d37a0d8146402a4baecbc0eef29df1bfa1513bddef445df8dea8f2c68cddf7b WatchSource:0}: Error finding container 6d37a0d8146402a4baecbc0eef29df1bfa1513bddef445df8dea8f2c68cddf7b: Status 404 returned error can't find the container with id 6d37a0d8146402a4baecbc0eef29df1bfa1513bddef445df8dea8f2c68cddf7b
Sep 11 18:42:46.501100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:46.501016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49" event={"ID":"a8d099b4-cf86-4998-b2bc-f243bc6e8ada","Type":"ContainerStarted","Data":"87e7a9132a1c0368c1ccdd640c55e670986616e92a90e10299b6220bc9550379"}
Sep 11 18:42:46.501100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:46.501075    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:42:46.501100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:46.501091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49" event={"ID":"a8d099b4-cf86-4998-b2bc-f243bc6e8ada","Type":"ContainerStarted","Data":"6d37a0d8146402a4baecbc0eef29df1bfa1513bddef445df8dea8f2c68cddf7b"}
Sep 11 18:42:46.519150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:46.519096    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49" podStartSLOduration=1.519080728 podStartE2EDuration="1.519080728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:42:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:42:46.516795641 +0000 UTC m=+1935.109996209" watchObservedRunningTime="2026-09-11 18:42:46.519080728 +0000 UTC m=+1935.112281285"
Sep 11 18:42:57.507811 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:42:57.507773    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:43:01.592450 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:01.592416    2563 generic.go:356] "Generic (PLEG): container finished" podID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerID="56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381" exitCode=0
Sep 11 18:43:01.592960 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:01.592498    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" event={"ID":"cfbe68d8-9410-46c4-8df3-2055548e6d0b","Type":"ContainerDied","Data":"56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"}
Sep 11 18:43:01.593035 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:01.593017    2563 scope.go:117] "RemoveContainer" containerID="56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"
Sep 11 18:43:01.594170 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:01.594142    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:43:01.594231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:01.594219    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:43:01.594292 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:01.594252    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 18:43:02.600150 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:02.600104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" event={"ID":"cfbe68d8-9410-46c4-8df3-2055548e6d0b","Type":"ContainerStarted","Data":"bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b"}
Sep 11 18:43:03.100228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:03.100187    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:43:03.100437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:03.100328    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 11 18:43:03.100437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:03.100373    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 11 18:43:04.100440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:04.100396    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 11 18:43:04.100954 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:04.100473    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 11 18:43:05.100975 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:05.100939    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 11 18:43:05.101541 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:05.101509    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 11 18:43:06.100997 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:06.100959    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 11 18:43:06.101613 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:06.101583    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 11 18:43:06.997727 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:06.997686    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:07.100918 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:07.100869    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Sep 11 18:43:07.101121 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:07.100940    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Sep 11 18:43:07.798845 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:07.798810    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:08.103911 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:08.103837    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:43:08.104292 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:08.104069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:43:08.104714 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:08.104694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:43:08.514836 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:08.514755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:09.170539 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:09.170505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:09.946819 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:09.946786    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:10.677453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:10.677422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:11.473777 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:11.473744    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:12.874377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:12.874344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:13.701046 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:13.701015    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:14.362230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:14.362201    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:15.183300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:15.183245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:15.963578 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:15.963546    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:16.626036 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:16.626003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:23.473284 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.472676    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:23.509115 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.509083    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"]
Sep 11 18:43:23.509385 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.509355    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49" podUID="a8d099b4-cf86-4998-b2bc-f243bc6e8ada" containerName="payload-processing" containerID="cri-o://87e7a9132a1c0368c1ccdd640c55e670986616e92a90e10299b6220bc9550379" gracePeriod=30
Sep 11 18:43:23.537713 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.537687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:23.595097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.595067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:23.682661 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.682633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:23.724893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.724821    2563 generic.go:356] "Generic (PLEG): container finished" podID="a8d099b4-cf86-4998-b2bc-f243bc6e8ada" containerID="87e7a9132a1c0368c1ccdd640c55e670986616e92a90e10299b6220bc9550379" exitCode=0
Sep 11 18:43:23.724893 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.724885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49" event={"ID":"a8d099b4-cf86-4998-b2bc-f243bc6e8ada","Type":"ContainerDied","Data":"87e7a9132a1c0368c1ccdd640c55e670986616e92a90e10299b6220bc9550379"}
Sep 11 18:43:23.755241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.755213    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:23.761914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.761895    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:43:23.894055 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.894021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8rd8\" (UniqueName: \"kubernetes.io/projected/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-kube-api-access-j8rd8\") pod \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") "
Sep 11 18:43:23.894227 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.894112    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-plugins-config-volume\") pod \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\" (UID: \"a8d099b4-cf86-4998-b2bc-f243bc6e8ada\") "
Sep 11 18:43:23.894523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.894490    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8d099b4-cf86-4998-b2bc-f243bc6e8ada" (UID: "a8d099b4-cf86-4998-b2bc-f243bc6e8ada"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:43:23.896166 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.896142    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-kube-api-access-j8rd8" (OuterVolumeSpecName: "kube-api-access-j8rd8") pod "a8d099b4-cf86-4998-b2bc-f243bc6e8ada" (UID: "a8d099b4-cf86-4998-b2bc-f243bc6e8ada"). InnerVolumeSpecName "kube-api-access-j8rd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:43:23.994976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.994912    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:23.994976 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:23.994936    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8rd8\" (UniqueName: \"kubernetes.io/projected/a8d099b4-cf86-4998-b2bc-f243bc6e8ada-kube-api-access-j8rd8\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:24.048936 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.048897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:24.152459 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.152422    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:24.732619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.732582    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"
Sep 11 18:43:24.732619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.732602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49" event={"ID":"a8d099b4-cf86-4998-b2bc-f243bc6e8ada","Type":"ContainerDied","Data":"6d37a0d8146402a4baecbc0eef29df1bfa1513bddef445df8dea8f2c68cddf7b"}
Sep 11 18:43:24.733097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.732700    2563 scope.go:117] "RemoveContainer" containerID="87e7a9132a1c0368c1ccdd640c55e670986616e92a90e10299b6220bc9550379"
Sep 11 18:43:24.753034 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.753002    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"]
Sep 11 18:43:24.763716 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.763692    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2146-6dd5598f67-xwn49"]
Sep 11 18:43:24.788100 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:24.788077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:25.471012 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:25.470981    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:25.913762 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:25.913732    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d099b4-cf86-4998-b2bc-f243bc6e8ada" path="/var/lib/kubelet/pods/a8d099b4-cf86-4998-b2bc-f243bc6e8ada/volumes"
Sep 11 18:43:26.281618 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:26.281592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:26.948482 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:26.948449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:30.116221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:30.116184    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:30.221305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:30.221244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:30.392748 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:30.392677    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"]
Sep 11 18:43:30.392917 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:30.392887    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy" containerID="cri-o://bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b" gracePeriod=30
Sep 11 18:43:30.421440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:30.421415    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:34.007491 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:34.007449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:34.896959 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:34.896928    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:35.653669 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.653644    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:43:35.704530 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704460    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istiod-ca-cert\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-envoy\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704676 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704565    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-certs\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-token\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704762    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-socket\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704790    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-credential-socket\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704827    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:43:35.704963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704846    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-podinfo\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704895    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-data\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.704963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704917    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:43:35.704963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.704944    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjtwb\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-kube-api-access-tjtwb\") pod \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\" (UID: \"cfbe68d8-9410-46c4-8df3-2055548e6d0b\") "
Sep 11 18:43:35.705213 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:43:35.705213 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705143    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:43:35.705407 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705388    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.705471 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705414    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.705471 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705428    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.705471 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705442    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.705607 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.705587    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:43:35.706966 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.706944    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:43:35.707386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.707364    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:43:35.707386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.707361    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:43:35.707553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.707531    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-kube-api-access-tjtwb" (OuterVolumeSpecName: "kube-api-access-tjtwb") pod "cfbe68d8-9410-46c4-8df3-2055548e6d0b" (UID: "cfbe68d8-9410-46c4-8df3-2055548e6d0b"). InnerVolumeSpecName "kube-api-access-tjtwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:43:35.712993 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.712971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:35.794232 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.794199    2563 generic.go:356] "Generic (PLEG): container finished" podID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerID="bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b" exitCode=0
Sep 11 18:43:35.794397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.794255    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"
Sep 11 18:43:35.794397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.794249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" event={"ID":"cfbe68d8-9410-46c4-8df3-2055548e6d0b","Type":"ContainerDied","Data":"bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b"}
Sep 11 18:43:35.794397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.794327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc" event={"ID":"cfbe68d8-9410-46c4-8df3-2055548e6d0b","Type":"ContainerDied","Data":"7dc1a40e0aefbcad20e7a33ff013aea7211aee082c6269bee454c7506d386662"}
Sep 11 18:43:35.794397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.794351    2563 scope.go:117] "RemoveContainer" containerID="bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b"
Sep 11 18:43:35.803801 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.803786    2563 scope.go:117] "RemoveContainer" containerID="56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"
Sep 11 18:43:35.806090 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.806069    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.806090 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.806089    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.806255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.806099    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjtwb\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-kube-api-access-tjtwb\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.806255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.806108    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.806255 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.806117    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfbe68d8-9410-46c4-8df3-2055548e6d0b-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:35.812475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.812458    2563 scope.go:117] "RemoveContainer" containerID="bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b"
Sep 11 18:43:35.812741 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:43:35.812721    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b\": container with ID starting with bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b not found: ID does not exist" containerID="bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b"
Sep 11 18:43:35.812790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.812750    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b"} err="failed to get container status \"bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b\": rpc error: code = NotFound desc = could not find container \"bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b\": container with ID starting with bc01cce41e9584b3e870812aff9f80ba494f8c1f112452312f39721dcf09299b not found: ID does not exist"
Sep 11 18:43:35.812790 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.812766    2563 scope.go:117] "RemoveContainer" containerID="56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"
Sep 11 18:43:35.812999 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:43:35.812980    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381\": container with ID starting with 56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381 not found: ID does not exist" containerID="56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"
Sep 11 18:43:35.813044 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.813005    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381"} err="failed to get container status \"56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381\": rpc error: code = NotFound desc = could not find container \"56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381\": container with ID starting with 56af65898eabacc6943aafb04d1f55b55fd6f1357b41c0a313623596f8d5d381 not found: ID does not exist"
Sep 11 18:43:35.820736 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.818459    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"]
Sep 11 18:43:35.825869 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.825847    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-cd2146-openshift-default-7c5789778d-jzhrc"]
Sep 11 18:43:35.913435 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:35.913406    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" path="/var/lib/kubelet/pods/cfbe68d8-9410-46c4-8df3-2055548e6d0b/volumes"
Sep 11 18:43:36.381029 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:36.380998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:37.037605 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:37.037558    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:37.809254 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:37.809221    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:49.372531 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.372491    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:49.397422 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.397393    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"]
Sep 11 18:43:49.397738 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.397705    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp" podUID="e6857185-862b-401b-af61-7cc744160028" containerName="payload-pre-processing" containerID="cri-o://7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929" gracePeriod=30
Sep 11 18:43:49.442898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.442871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:49.544716 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.544667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:49.612687 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.612643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:49.664424 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.664397    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:43:49.738405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.737567    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6857185-862b-401b-af61-7cc744160028-plugins-config-volume\") pod \"e6857185-862b-401b-af61-7cc744160028\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") "
Sep 11 18:43:49.738405 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.737660    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srf5r\" (UniqueName: \"kubernetes.io/projected/e6857185-862b-401b-af61-7cc744160028-kube-api-access-srf5r\") pod \"e6857185-862b-401b-af61-7cc744160028\" (UID: \"e6857185-862b-401b-af61-7cc744160028\") "
Sep 11 18:43:49.738717 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.738686    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6857185-862b-401b-af61-7cc744160028-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6857185-862b-401b-af61-7cc744160028" (UID: "e6857185-862b-401b-af61-7cc744160028"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:43:49.740342 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.740304    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6857185-862b-401b-af61-7cc744160028-kube-api-access-srf5r" (OuterVolumeSpecName: "kube-api-access-srf5r") pod "e6857185-862b-401b-af61-7cc744160028" (UID: "e6857185-862b-401b-af61-7cc744160028"). InnerVolumeSpecName "kube-api-access-srf5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:43:49.839083 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.839050    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6857185-862b-401b-af61-7cc744160028-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:49.839083 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.839077    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srf5r\" (UniqueName: \"kubernetes.io/projected/e6857185-862b-401b-af61-7cc744160028-kube-api-access-srf5r\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:43:49.866303 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.866245    2563 generic.go:356] "Generic (PLEG): container finished" podID="e6857185-862b-401b-af61-7cc744160028" containerID="7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929" exitCode=0
Sep 11 18:43:49.866453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.866324    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"
Sep 11 18:43:49.866453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.866330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp" event={"ID":"e6857185-862b-401b-af61-7cc744160028","Type":"ContainerDied","Data":"7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929"}
Sep 11 18:43:49.866453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.866384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp" event={"ID":"e6857185-862b-401b-af61-7cc744160028","Type":"ContainerDied","Data":"e38d241e3b1d45d57b3990cb52df1be2962cbdaaf811e9eb13377718c2ffba19"}
Sep 11 18:43:49.866453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.866406    2563 scope.go:117] "RemoveContainer" containerID="7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929"
Sep 11 18:43:49.875348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.875331    2563 scope.go:117] "RemoveContainer" containerID="7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929"
Sep 11 18:43:49.875603 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:43:49.875582    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929\": container with ID starting with 7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929 not found: ID does not exist" containerID="7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929"
Sep 11 18:43:49.875671 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.875613    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929"} err="failed to get container status \"7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929\": rpc error: code = NotFound desc = could not find container \"7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929\": container with ID starting with 7e1d1477f091e9791a6ff593849cba5371dd8725bb8338096441c784f21df929 not found: ID does not exist"
Sep 11 18:43:49.887706 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.887683    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"]
Sep 11 18:43:49.891574 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.891522    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-765dd1-7c5586f698-pfrhp"]
Sep 11 18:43:49.912585 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.912561    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6857185-862b-401b-af61-7cc744160028" path="/var/lib/kubelet/pods/e6857185-862b-401b-af61-7cc744160028/volumes"
Sep 11 18:43:49.953914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:49.953885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:50.050963 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:50.050935    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:50.772566 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:50.772535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:51.692814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:51.692780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:52.390652 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:52.390619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:43:54.918071 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:54.918040    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"]
Sep 11 18:43:54.964523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:43:54.964483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:08.977378 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:08.977341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:09.937076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:09.937012    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw" podUID="d296c813-1fb4-44c4-ad1e-812d14eefd96" containerName="main" containerID="cri-o://4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616" gracePeriod=45
Sep 11 18:44:10.183727 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.183703    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:44:10.217757 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.217689    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d296c813-1fb4-44c4-ad1e-812d14eefd96-tls-certs\") pod \"d296c813-1fb4-44c4-ad1e-812d14eefd96\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") "
Sep 11 18:44:10.217894 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.217779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-tmp-dir\") pod \"d296c813-1fb4-44c4-ad1e-812d14eefd96\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") "
Sep 11 18:44:10.217894 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.217811    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-dshm\") pod \"d296c813-1fb4-44c4-ad1e-812d14eefd96\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") "
Sep 11 18:44:10.217894 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.217881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc49q\" (UniqueName: \"kubernetes.io/projected/d296c813-1fb4-44c4-ad1e-812d14eefd96-kube-api-access-wc49q\") pod \"d296c813-1fb4-44c4-ad1e-812d14eefd96\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") "
Sep 11 18:44:10.218059 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.217909    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-home\") pod \"d296c813-1fb4-44c4-ad1e-812d14eefd96\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") "
Sep 11 18:44:10.218059 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.218023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-model-cache\") pod \"d296c813-1fb4-44c4-ad1e-812d14eefd96\" (UID: \"d296c813-1fb4-44c4-ad1e-812d14eefd96\") "
Sep 11 18:44:10.218477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.218245    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-home" (OuterVolumeSpecName: "home") pod "d296c813-1fb4-44c4-ad1e-812d14eefd96" (UID: "d296c813-1fb4-44c4-ad1e-812d14eefd96"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:10.218477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.218390    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-home\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:10.218477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.218398    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-model-cache" (OuterVolumeSpecName: "model-cache") pod "d296c813-1fb4-44c4-ad1e-812d14eefd96" (UID: "d296c813-1fb4-44c4-ad1e-812d14eefd96"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:10.218477 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.218451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d296c813-1fb4-44c4-ad1e-812d14eefd96" (UID: "d296c813-1fb4-44c4-ad1e-812d14eefd96"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:10.220090 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.220062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d296c813-1fb4-44c4-ad1e-812d14eefd96-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d296c813-1fb4-44c4-ad1e-812d14eefd96" (UID: "d296c813-1fb4-44c4-ad1e-812d14eefd96"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:44:10.220208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.220124    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d296c813-1fb4-44c4-ad1e-812d14eefd96-kube-api-access-wc49q" (OuterVolumeSpecName: "kube-api-access-wc49q") pod "d296c813-1fb4-44c4-ad1e-812d14eefd96" (UID: "d296c813-1fb4-44c4-ad1e-812d14eefd96"). InnerVolumeSpecName "kube-api-access-wc49q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:44:10.220443 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.220426    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-dshm" (OuterVolumeSpecName: "dshm") pod "d296c813-1fb4-44c4-ad1e-812d14eefd96" (UID: "d296c813-1fb4-44c4-ad1e-812d14eefd96"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:10.319089 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.319064    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-tmp-dir\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:10.319089 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.319088    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-dshm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:10.319241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.319100    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc49q\" (UniqueName: \"kubernetes.io/projected/d296c813-1fb4-44c4-ad1e-812d14eefd96-kube-api-access-wc49q\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:10.319241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.319112    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d296c813-1fb4-44c4-ad1e-812d14eefd96-model-cache\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:10.319241 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.319120    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d296c813-1fb4-44c4-ad1e-812d14eefd96-tls-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:10.982002 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.981964    2563 generic.go:356] "Generic (PLEG): container finished" podID="d296c813-1fb4-44c4-ad1e-812d14eefd96" containerID="4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616" exitCode=0
Sep 11 18:44:10.982196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.982011    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"
Sep 11 18:44:10.982196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.982048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw" event={"ID":"d296c813-1fb4-44c4-ad1e-812d14eefd96","Type":"ContainerDied","Data":"4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616"}
Sep 11 18:44:10.982196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.982091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw" event={"ID":"d296c813-1fb4-44c4-ad1e-812d14eefd96","Type":"ContainerDied","Data":"e949b7f127600e1004c00c34389fc13acf0d7b1007cf14ffe6e04cb28581f23f"}
Sep 11 18:44:10.982196 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.982109    2563 scope.go:117] "RemoveContainer" containerID="4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616"
Sep 11 18:44:10.991554 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.991539    2563 scope.go:117] "RemoveContainer" containerID="4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616"
Sep 11 18:44:10.991797 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:44:10.991779    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616\": container with ID starting with 4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616 not found: ID does not exist" containerID="4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616"
Sep 11 18:44:10.991849 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:10.991804    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616"} err="failed to get container status \"4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616\": rpc error: code = NotFound desc = could not find container \"4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616\": container with ID starting with 4d35268791605b3254858a1d7cb6c31984a19b7ef23b5ec7c3a7274bb6544616 not found: ID does not exist"
Sep 11 18:44:11.004324 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:11.004299    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"]
Sep 11 18:44:11.007937 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:11.007916    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-765dd1/ipp-route-765dd1-kserve-78757948c9-p7kfw"]
Sep 11 18:44:11.913364 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:11.913323    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d296c813-1fb4-44c4-ad1e-812d14eefd96" path="/var/lib/kubelet/pods/d296c813-1fb4-44c4-ad1e-812d14eefd96/volumes"
Sep 11 18:44:19.087097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:19.087067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:31.904048 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:31.904016    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:33.043119 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:33.043071    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:38.924967 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:38.924911    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:38.981098 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:38.981072    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:39.183123 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.183037    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"]
Sep 11 18:44:39.183598 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.183567    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy" containerID="cri-o://e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47" gracePeriod=30
Sep 11 18:44:39.197624 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.197595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:39.548233 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548205    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"]
Sep 11 18:44:39.548662 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548648    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d296c813-1fb4-44c4-ad1e-812d14eefd96" containerName="main"
Sep 11 18:44:39.548709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548663    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d296c813-1fb4-44c4-ad1e-812d14eefd96" containerName="main"
Sep 11 18:44:39.548709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548673    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy"
Sep 11 18:44:39.548709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548679    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy"
Sep 11 18:44:39.548709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548687    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy"
Sep 11 18:44:39.548709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548693    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy"
Sep 11 18:44:39.548709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548707    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6857185-862b-401b-af61-7cc744160028" containerName="payload-pre-processing"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548713    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6857185-862b-401b-af61-7cc744160028" containerName="payload-pre-processing"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548720    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d099b4-cf86-4998-b2bc-f243bc6e8ada" containerName="payload-processing"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548725    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d099b4-cf86-4998-b2bc-f243bc6e8ada" containerName="payload-processing"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548783    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d296c813-1fb4-44c4-ad1e-812d14eefd96" containerName="main"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548791    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548798    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d099b4-cf86-4998-b2bc-f243bc6e8ada" containerName="payload-processing"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548806    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbe68d8-9410-46c4-8df3-2055548e6d0b" containerName="istio-proxy"
Sep 11 18:44:39.548898 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.548811    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6857185-862b-401b-af61-7cc744160028" containerName="payload-pre-processing"
Sep 11 18:44:39.551579 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.551554    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:39.553805 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.553786    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t9kdx\""
Sep 11 18:44:39.561432 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.561405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"]
Sep 11 18:44:39.678832 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.678799    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebaa784-623f-4156-a145-5fc07ac5ae45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj\" (UID: \"7ebaa784-623f-4156-a145-5fc07ac5ae45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:39.688094 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.688022    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:39.780310 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.780244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebaa784-623f-4156-a145-5fc07ac5ae45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj\" (UID: \"7ebaa784-623f-4156-a145-5fc07ac5ae45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:39.780904 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.780874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebaa784-623f-4156-a145-5fc07ac5ae45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj\" (UID: \"7ebaa784-623f-4156-a145-5fc07ac5ae45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:39.863018 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:39.862937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:40.144530 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:40.144507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"]
Sep 11 18:44:40.145929 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:44:40.145902    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebaa784_623f_4156_a145_5fc07ac5ae45.slice/crio-c3ca2996fcfc2111f94997c2cb8e095252cb4a4489b9d2a35b330da59aef211f WatchSource:0}: Error finding container c3ca2996fcfc2111f94997c2cb8e095252cb4a4489b9d2a35b330da59aef211f: Status 404 returned error can't find the container with id c3ca2996fcfc2111f94997c2cb8e095252cb4a4489b9d2a35b330da59aef211f
Sep 11 18:44:40.644202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:40.644167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:41.138553 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:41.138516    2563 generic.go:356] "Generic (PLEG): container finished" podID="7ebaa784-623f-4156-a145-5fc07ac5ae45" containerID="84cb1453e17ac6c20d2a1b15aca3bbf1c24ae25f9668f32fbdd13c18a0fb3e6d" exitCode=0
Sep 11 18:44:41.138709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:41.138562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj" event={"ID":"7ebaa784-623f-4156-a145-5fc07ac5ae45","Type":"ContainerDied","Data":"84cb1453e17ac6c20d2a1b15aca3bbf1c24ae25f9668f32fbdd13c18a0fb3e6d"}
Sep 11 18:44:41.138709 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:41.138615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj" event={"ID":"7ebaa784-623f-4156-a145-5fc07ac5ae45","Type":"ContainerStarted","Data":"c3ca2996fcfc2111f94997c2cb8e095252cb4a4489b9d2a35b330da59aef211f"}
Sep 11 18:44:41.390593 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:41.390505    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:42.089705 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:42.089679    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:42.276215 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:42.276193    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:42.406839 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:42.406771    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebaa784-623f-4156-a145-5fc07ac5ae45-maas-api-service-ca\") pod \"7ebaa784-623f-4156-a145-5fc07ac5ae45\" (UID: \"7ebaa784-623f-4156-a145-5fc07ac5ae45\") "
Sep 11 18:44:42.407153 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:42.407138    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ebaa784-623f-4156-a145-5fc07ac5ae45-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ebaa784-623f-4156-a145-5fc07ac5ae45" (UID: "7ebaa784-623f-4156-a145-5fc07ac5ae45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:44:42.407239 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:42.407224    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ebaa784-623f-4156-a145-5fc07ac5ae45-maas-api-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:42.886606 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:42.886565    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:43.149619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:43.149550    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"
Sep 11 18:44:43.149619 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:43.149574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj" event={"ID":"7ebaa784-623f-4156-a145-5fc07ac5ae45","Type":"ContainerDied","Data":"c3ca2996fcfc2111f94997c2cb8e095252cb4a4489b9d2a35b330da59aef211f"}
Sep 11 18:44:43.149809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:43.149622    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3ca2996fcfc2111f94997c2cb8e095252cb4a4489b9d2a35b330da59aef211f"
Sep 11 18:44:43.608166 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:43.608139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:44.438356 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.438335    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:44:44.525019 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.524997    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-certs\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525075    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-envoy\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525137 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525102    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-socket\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-credential-socket\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525221 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brjb8\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-kube-api-access-brjb8\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525234    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istiod-ca-cert\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525280    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:44.525337 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525316    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-data\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525371    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-token\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525398    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-podinfo\") pod \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\" (UID: \"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40\") "
Sep 11 18:44:44.525497 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:44.525646 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:44.525701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525556    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-data" (OuterVolumeSpecName: "istio-data") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:44.525701 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525685    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:44:44.525825 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525806    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.525890 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525829    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.525890 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525841    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.525890 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525850    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.525890 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.525860    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.527209 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.527189    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:44:44.527677 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.527656    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:44:44.527903 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.527883    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-kube-api-access-brjb8" (OuterVolumeSpecName: "kube-api-access-brjb8") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "kube-api-access-brjb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:44:44.527956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.527923    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-token" (OuterVolumeSpecName: "istio-token") pod "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" (UID: "3fc8e855-11a4-4ffc-b116-6b06aa1cfd40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:44:44.627009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.626982    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.627009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.627004    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.627009 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.627013    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:44.627385 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:44.627023    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brjb8\" (UniqueName: \"kubernetes.io/projected/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40-kube-api-access-brjb8\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:44:45.160300 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.160268    2563 generic.go:356] "Generic (PLEG): container finished" podID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerID="e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47" exitCode=0
Sep 11 18:44:45.160451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.160316    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"
Sep 11 18:44:45.160451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.160349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" event={"ID":"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40","Type":"ContainerDied","Data":"e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47"}
Sep 11 18:44:45.160451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.160404    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd" event={"ID":"3fc8e855-11a4-4ffc-b116-6b06aa1cfd40","Type":"ContainerDied","Data":"ad9ed6b6cfd2075f556ef75d4b1f84ec7bf8e0e212d73a76edc59f972dd7892e"}
Sep 11 18:44:45.160451 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.160426    2563 scope.go:117] "RemoveContainer" containerID="e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47"
Sep 11 18:44:45.176347 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.176330    2563 scope.go:117] "RemoveContainer" containerID="1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"
Sep 11 18:44:45.184978 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.184957    2563 scope.go:117] "RemoveContainer" containerID="e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47"
Sep 11 18:44:45.185243 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:44:45.185223    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47\": container with ID starting with e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47 not found: ID does not exist" containerID="e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47"
Sep 11 18:44:45.185297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.185252    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47"} err="failed to get container status \"e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47\": rpc error: code = NotFound desc = could not find container \"e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47\": container with ID starting with e85d96d8775bf36813e51c3c06279167855337e0b7f9977a255403bbe46aba47 not found: ID does not exist"
Sep 11 18:44:45.185297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.185292    2563 scope.go:117] "RemoveContainer" containerID="1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"
Sep 11 18:44:45.185560 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:44:45.185540    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606\": container with ID starting with 1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606 not found: ID does not exist" containerID="1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"
Sep 11 18:44:45.185614 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.185564    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606"} err="failed to get container status \"1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606\": rpc error: code = NotFound desc = could not find container \"1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606\": container with ID starting with 1468389a28bba318d2ec72b291edeeca8f3666b3514a0b67e8bbf22ae8ed1606 not found: ID does not exist"
Sep 11 18:44:45.189289 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.189256    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"]
Sep 11 18:44:45.193398 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.193376    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-765dd1-openshift-default-596d84cc9b-bdgdd"]
Sep 11 18:44:45.913294 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:45.913242    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" path="/var/lib/kubelet/pods/3fc8e855-11a4-4ffc-b116-6b06aa1cfd40/volumes"
Sep 11 18:44:50.070696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.070665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:50.155225 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.155196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:50.234680 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.234648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:50.295441 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.295413    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"]
Sep 11 18:44:50.299208 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.299178    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-9c1a4f-zx8mj"]
Sep 11 18:44:50.307156 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.307136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:50.715252 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.715218    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:50.813433 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:50.813404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:51.581437 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:51.581391    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:51.913515 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:51.913434    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ebaa784-623f-4156-a145-5fc07ac5ae45" path="/var/lib/kubelet/pods/7ebaa784-623f-4156-a145-5fc07ac5ae45/volumes"
Sep 11 18:44:52.289344 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:52.289306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:56.674525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:56.674434    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:56.736065 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:56.736019    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:56.933217 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:56.933132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:57.504388 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:57.504360    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:58.240523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:58.240473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:58.953193 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:58.953157    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:44:59.615282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:44:59.615238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:00.135995 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.135963    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"]
Sep 11 18:45:00.136442 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136427    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy"
Sep 11 18:45:00.136510 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136444    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy"
Sep 11 18:45:00.136510 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136459    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebaa784-623f-4156-a145-5fc07ac5ae45" containerName="revoke-keys"
Sep 11 18:45:00.136510 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136465    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebaa784-623f-4156-a145-5fc07ac5ae45" containerName="revoke-keys"
Sep 11 18:45:00.136628 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136525    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ebaa784-623f-4156-a145-5fc07ac5ae45" containerName="revoke-keys"
Sep 11 18:45:00.136628 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136535    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy"
Sep 11 18:45:00.136628 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.136542    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy"
Sep 11 18:45:00.140998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.140980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"
Sep 11 18:45:00.143596 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.143576    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t9kdx\""
Sep 11 18:45:00.149043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.149015    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"]
Sep 11 18:45:00.153956 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.153935    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"
Sep 11 18:45:00.435001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:00.434979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"]
Sep 11 18:45:00.436443 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:45:00.436415    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76030a9_5f07_46a5_a53a_44934691be4a.slice/crio-f14e606612b6e38594c8a0e6e69e0f62bc5aed7bfd68c830d8bef700b4f56420 WatchSource:0}: Error finding container f14e606612b6e38594c8a0e6e69e0f62bc5aed7bfd68c830d8bef700b4f56420: Status 404 returned error can't find the container with id f14e606612b6e38594c8a0e6e69e0f62bc5aed7bfd68c830d8bef700b4f56420
Sep 11 18:45:01.245088 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:01.245059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx" event={"ID":"d76030a9-5f07-46a5-a53a-44934691be4a","Type":"ContainerStarted","Data":"f7d30994e9495d95b1f42bdf22425e5ce6619984f8e752bced79401500d5ae52"}
Sep 11 18:45:01.245470 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:01.245106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx" event={"ID":"d76030a9-5f07-46a5-a53a-44934691be4a","Type":"ContainerStarted","Data":"f14e606612b6e38594c8a0e6e69e0f62bc5aed7bfd68c830d8bef700b4f56420"}
Sep 11 18:45:01.260635 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:01.260580    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx" podStartSLOduration=1.260565341 podStartE2EDuration="1.260565341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:45:01.25887235 +0000 UTC m=+2069.852072907" watchObservedRunningTime="2026-09-11 18:45:01.260565341 +0000 UTC m=+2069.853765898"
Sep 11 18:45:02.251174 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:02.251138    2563 generic.go:356] "Generic (PLEG): container finished" podID="d76030a9-5f07-46a5-a53a-44934691be4a" containerID="f7d30994e9495d95b1f42bdf22425e5ce6619984f8e752bced79401500d5ae52" exitCode=0
Sep 11 18:45:02.251617 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:02.251226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx" event={"ID":"d76030a9-5f07-46a5-a53a-44934691be4a","Type":"ContainerDied","Data":"f7d30994e9495d95b1f42bdf22425e5ce6619984f8e752bced79401500d5ae52"}
Sep 11 18:45:03.383318 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:03.383287    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"
Sep 11 18:45:04.263207 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:04.263123    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"
Sep 11 18:45:04.263207 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:04.263162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx" event={"ID":"d76030a9-5f07-46a5-a53a-44934691be4a","Type":"ContainerDied","Data":"f14e606612b6e38594c8a0e6e69e0f62bc5aed7bfd68c830d8bef700b4f56420"}
Sep 11 18:45:04.263449 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:04.263245    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f14e606612b6e38594c8a0e6e69e0f62bc5aed7bfd68c830d8bef700b4f56420"
Sep 11 18:45:05.223631 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:05.223592    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:05.278998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:05.278971    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:07.846588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.846543    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:07.850971 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.850945    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"]
Sep 11 18:45:07.853024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.853001    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-bf9898-29819205-rs7gx"]
Sep 11 18:45:07.871370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.871345    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"]
Sep 11 18:45:07.871635 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.871603    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6" podUID="8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" containerName="payload-processing" containerID="cri-o://c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962" gracePeriod=30
Sep 11 18:45:07.913740 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.913715    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d76030a9-5f07-46a5-a53a-44934691be4a" path="/var/lib/kubelet/pods/d76030a9-5f07-46a5-a53a-44934691be4a/volumes"
Sep 11 18:45:07.945736 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:07.945714    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:08.039374 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.039347    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:08.116225 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.116203    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:45:08.234770 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.234740    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpdg4\" (UniqueName: \"kubernetes.io/projected/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-kube-api-access-tpdg4\") pod \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") "
Sep 11 18:45:08.234974 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.234831    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-plugins-config-volume\") pod \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\" (UID: \"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99\") "
Sep 11 18:45:08.235205 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.235179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" (UID: "8833bc1b-04fb-4f06-9af4-6b6ff28e4c99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:08.236978 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.236941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-kube-api-access-tpdg4" (OuterVolumeSpecName: "kube-api-access-tpdg4") pod "8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" (UID: "8833bc1b-04fb-4f06-9af4-6b6ff28e4c99"). InnerVolumeSpecName "kube-api-access-tpdg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:08.285216 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.285192    2563 generic.go:356] "Generic (PLEG): container finished" podID="8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" containerID="c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962" exitCode=0
Sep 11 18:45:08.285348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.285245    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"
Sep 11 18:45:08.285348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.285290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6" event={"ID":"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99","Type":"ContainerDied","Data":"c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962"}
Sep 11 18:45:08.285348 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.285342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6" event={"ID":"8833bc1b-04fb-4f06-9af4-6b6ff28e4c99","Type":"ContainerDied","Data":"dafb4298fd86c4c03f49dcaf722db37c0d7274cfe64ce9766d0dfeb485d67260"}
Sep 11 18:45:08.285505 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.285369    2563 scope.go:117] "RemoveContainer" containerID="c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962"
Sep 11 18:45:08.294049 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.294033    2563 scope.go:117] "RemoveContainer" containerID="c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962"
Sep 11 18:45:08.294316 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:45:08.294296    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962\": container with ID starting with c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962 not found: ID does not exist" containerID="c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962"
Sep 11 18:45:08.294372 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.294325    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962"} err="failed to get container status \"c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962\": rpc error: code = NotFound desc = could not find container \"c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962\": container with ID starting with c3ca592386ffc8630adb1249bfabe4b9ecdd7e626edeaa50423f8b2617be6962 not found: ID does not exist"
Sep 11 18:45:08.307682 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.307658    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"]
Sep 11 18:45:08.311220 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.311202    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-bf9898-6789457b87-c6rm6"]
Sep 11 18:45:08.335611 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.335591    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-plugins-config-volume\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:08.335698 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.335613    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpdg4\" (UniqueName: \"kubernetes.io/projected/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99-kube-api-access-tpdg4\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:08.807377 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.807344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:08.926223 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:08.926193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:09.200151 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:09.200077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:09.919172 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:09.919138    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" path="/var/lib/kubelet/pods/8833bc1b-04fb-4f06-9af4-6b6ff28e4c99/volumes"
Sep 11 18:45:14.509681 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:14.509645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:14.573843 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:14.573815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:14.758710 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:14.758633    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"]
Sep 11 18:45:14.758873 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:14.758839    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy" containerID="cri-o://ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00" gracePeriod=30
Sep 11 18:45:14.775694 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:14.775671    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:15.783998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:15.783947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:16.579675 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:16.579642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:20.006803 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.006777    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:45:20.140634 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140593    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51468e39-25ed-4574-9768-3fe097f31a53-istio-podinfo\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.140787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140660    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-istio-token\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.140787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140681    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-envoy\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.140787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140718    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-certs\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.140787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140742    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-credential-socket\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.140787 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nppx5\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-kube-api-access-nppx5\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.141040 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-data\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.141040 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140822    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51468e39-25ed-4574-9768-3fe097f31a53-istiod-ca-cert\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.141040 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.140923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-socket\") pod \"51468e39-25ed-4574-9768-3fe097f31a53\" (UID: \"51468e39-25ed-4574-9768-3fe097f31a53\") "
Sep 11 18:45:20.141040 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141025    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:20.141235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141146    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-data" (OuterVolumeSpecName: "istio-data") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:20.141235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:20.141235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141224    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:20.141434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141359    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.141434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141381    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.141434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141394    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.141593 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.141428    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51468e39-25ed-4574-9768-3fe097f31a53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:20.142905 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.142881    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-istio-token" (OuterVolumeSpecName: "istio-token") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:20.143027 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.142969    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:20.143121 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.143106    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51468e39-25ed-4574-9768-3fe097f31a53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:45:20.143162 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.143125    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-kube-api-access-nppx5" (OuterVolumeSpecName: "kube-api-access-nppx5") pod "51468e39-25ed-4574-9768-3fe097f31a53" (UID: "51468e39-25ed-4574-9768-3fe097f31a53"). InnerVolumeSpecName "kube-api-access-nppx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:20.242320 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.242298    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.242320 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.242318    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.242457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.242327    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51468e39-25ed-4574-9768-3fe097f31a53-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.242457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.242337    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nppx5\" (UniqueName: \"kubernetes.io/projected/51468e39-25ed-4574-9768-3fe097f31a53-kube-api-access-nppx5\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.242457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.242345    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51468e39-25ed-4574-9768-3fe097f31a53-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.242457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.242353    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51468e39-25ed-4574-9768-3fe097f31a53-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:20.356039 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.356012    2563 generic.go:356] "Generic (PLEG): container finished" podID="51468e39-25ed-4574-9768-3fe097f31a53" containerID="ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00" exitCode=0
Sep 11 18:45:20.356155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.356056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"
Sep 11 18:45:20.356155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.356097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" event={"ID":"51468e39-25ed-4574-9768-3fe097f31a53","Type":"ContainerDied","Data":"ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00"}
Sep 11 18:45:20.356155 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.356152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm" event={"ID":"51468e39-25ed-4574-9768-3fe097f31a53","Type":"ContainerDied","Data":"54dcfadca859013dd8127619a621737b580a23d0bd2ab605111f3e366e7d0d42"}
Sep 11 18:45:20.356287 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.356175    2563 scope.go:117] "RemoveContainer" containerID="ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00"
Sep 11 18:45:20.365511 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.365493    2563 scope.go:117] "RemoveContainer" containerID="0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"
Sep 11 18:45:20.373506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.373490    2563 scope.go:117] "RemoveContainer" containerID="ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00"
Sep 11 18:45:20.373775 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:45:20.373753    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00\": container with ID starting with ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00 not found: ID does not exist" containerID="ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00"
Sep 11 18:45:20.373835 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.373781    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00"} err="failed to get container status \"ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00\": rpc error: code = NotFound desc = could not find container \"ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00\": container with ID starting with ccc4133230d69b5d4cacf77b46eda8fe258f4b0bcf0fe77895a5d19bad4a2b00 not found: ID does not exist"
Sep 11 18:45:20.373835 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.373797    2563 scope.go:117] "RemoveContainer" containerID="0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"
Sep 11 18:45:20.374044 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:45:20.374027    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821\": container with ID starting with 0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821 not found: ID does not exist" containerID="0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"
Sep 11 18:45:20.374088 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.374050    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821"} err="failed to get container status \"0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821\": rpc error: code = NotFound desc = could not find container \"0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821\": container with ID starting with 0ae8b3c87e19464e7b30632f4436a945dec80d86546990896a2b23ef9303c821 not found: ID does not exist"
Sep 11 18:45:20.396509 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.396458    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"]
Sep 11 18:45:20.401455 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:20.401435    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-bf9898-openshift-default-fc9854c76-9nmpm"]
Sep 11 18:45:21.913704 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:21.913676    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51468e39-25ed-4574-9768-3fe097f31a53" path="/var/lib/kubelet/pods/51468e39-25ed-4574-9768-3fe097f31a53/volumes"
Sep 11 18:45:25.570363 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:25.570316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:25.640922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:25.640885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:25.701779 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:25.701751    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:25.779930 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:25.779898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:25.852229 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:25.852164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:26.146387 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:26.146314    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:26.290072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:26.290041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:32.050768 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.050741    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:45:32.058561 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.058536    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:45:32.148741 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.148713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:32.248651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.248627    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:32.411851 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.411775    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"]
Sep 11 18:45:32.412004 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.411981    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy" containerID="cri-o://1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6" gracePeriod=30
Sep 11 18:45:32.460236 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:32.460210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:45:34.966875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:34.966837    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-556697db7-qfjsc"]
Sep 11 18:45:34.967524 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:34.967019    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-556697db7-qfjsc" podUID="71633f76-16bd-4065-8521-af0bdb28a5d1" containerName="manager" containerID="cri-o://6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44" gracePeriod=30
Sep 11 18:45:35.315185 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.315159    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:45:35.372949 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.372918    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs\") pod \"71633f76-16bd-4065-8521-af0bdb28a5d1\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") "
Sep 11 18:45:35.373110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.373097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7zbm\" (UniqueName: \"kubernetes.io/projected/71633f76-16bd-4065-8521-af0bdb28a5d1-kube-api-access-s7zbm\") pod \"71633f76-16bd-4065-8521-af0bdb28a5d1\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") "
Sep 11 18:45:35.373179 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.373157    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert\") pod \"71633f76-16bd-4065-8521-af0bdb28a5d1\" (UID: \"71633f76-16bd-4065-8521-af0bdb28a5d1\") "
Sep 11 18:45:35.374914 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.374888    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "71633f76-16bd-4065-8521-af0bdb28a5d1" (UID: "71633f76-16bd-4065-8521-af0bdb28a5d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:45:35.375131 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.375108    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71633f76-16bd-4065-8521-af0bdb28a5d1-kube-api-access-s7zbm" (OuterVolumeSpecName: "kube-api-access-s7zbm") pod "71633f76-16bd-4065-8521-af0bdb28a5d1" (UID: "71633f76-16bd-4065-8521-af0bdb28a5d1"). InnerVolumeSpecName "kube-api-access-s7zbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:35.375206 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.375115    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "71633f76-16bd-4065-8521-af0bdb28a5d1" (UID: "71633f76-16bd-4065-8521-af0bdb28a5d1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:45:35.434996 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.434969    2563 generic.go:356] "Generic (PLEG): container finished" podID="71633f76-16bd-4065-8521-af0bdb28a5d1" containerID="6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44" exitCode=0
Sep 11 18:45:35.435109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.435026    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-qfjsc"
Sep 11 18:45:35.435109 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.435029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-qfjsc" event={"ID":"71633f76-16bd-4065-8521-af0bdb28a5d1","Type":"ContainerDied","Data":"6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44"}
Sep 11 18:45:35.435228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.435114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-qfjsc" event={"ID":"71633f76-16bd-4065-8521-af0bdb28a5d1","Type":"ContainerDied","Data":"631edab59de8fa9392b8d4192cb8c1f1f80c3d774f8665020ec6b60fbc10f787"}
Sep 11 18:45:35.435228 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.435149    2563 scope.go:117] "RemoveContainer" containerID="6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44"
Sep 11 18:45:35.445416 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.445400    2563 scope.go:117] "RemoveContainer" containerID="6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44"
Sep 11 18:45:35.445696 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:45:35.445678    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44\": container with ID starting with 6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44 not found: ID does not exist" containerID="6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44"
Sep 11 18:45:35.445744 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.445707    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44"} err="failed to get container status \"6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44\": rpc error: code = NotFound desc = could not find container \"6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44\": container with ID starting with 6a879ee9bff4a7bcd7f1eab76945fa227d7f6b1de5e82c67d58180b3ecf0bb44 not found: ID does not exist"
Sep 11 18:45:35.457128 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.457103    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-556697db7-qfjsc"]
Sep 11 18:45:35.460181 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.460161    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-556697db7-qfjsc"]
Sep 11 18:45:35.474296 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.474275    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-webhook-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:35.474362 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.474299    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71633f76-16bd-4065-8521-af0bdb28a5d1-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:35.474362 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.474313    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7zbm\" (UniqueName: \"kubernetes.io/projected/71633f76-16bd-4065-8521-af0bdb28a5d1-kube-api-access-s7zbm\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:35.912867 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:35.912837    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71633f76-16bd-4065-8521-af0bdb28a5d1" path="/var/lib/kubelet/pods/71633f76-16bd-4065-8521-af0bdb28a5d1/volumes"
Sep 11 18:45:37.682638 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.682591    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:45:37.695125 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-envoy\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9c84\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-kube-api-access-z9c84\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695185    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-token\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695203    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istiod-ca-cert\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695246 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-podinfo\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695283    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-data\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695318    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-certs\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-socket\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695493 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695381    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-credential-socket\") pod \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\" (UID: \"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66\") "
Sep 11 18:45:37.695699 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695641    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:37.695699 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695677    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:45:37.695815 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695776    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istiod-ca-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.695871 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695837    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.695927 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695887    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:37.696022 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.695998    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:37.696081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.696063    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:37.697467 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.697443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-kube-api-access-z9c84" (OuterVolumeSpecName: "kube-api-access-z9c84") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "kube-api-access-z9c84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:37.697576 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.697474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:45:37.697858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.697837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:45:37.697858 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.697844    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" (UID: "8f0702ba-9a7e-452f-a7d4-a363bdbd2c66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:45:37.796584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796559    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-envoy\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.796584 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796582    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9c84\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-kube-api-access-z9c84\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.796723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796592    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-token\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.796723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796600    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-podinfo\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.796723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796616    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-istio-data\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.796723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796625    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-workload-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:37.796723 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:37.796634    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66-credential-socket\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:45:38.452665 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.452637    2563 generic.go:356] "Generic (PLEG): container finished" podID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerID="1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6" exitCode=0
Sep 11 18:45:38.452826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.452683    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"
Sep 11 18:45:38.452826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.452724    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" event={"ID":"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66","Type":"ContainerDied","Data":"1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6"}
Sep 11 18:45:38.452826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.452779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw" event={"ID":"8f0702ba-9a7e-452f-a7d4-a363bdbd2c66","Type":"ContainerDied","Data":"113386a8ddbb6baa5143fc8c8c3cd1d27c364127fb26572b3b423346f7b039cb"}
Sep 11 18:45:38.452826 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.452801    2563 scope.go:117] "RemoveContainer" containerID="1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6"
Sep 11 18:45:38.461305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.461255    2563 scope.go:117] "RemoveContainer" containerID="58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"
Sep 11 18:45:38.468970 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.468954    2563 scope.go:117] "RemoveContainer" containerID="1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6"
Sep 11 18:45:38.469223 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:45:38.469204    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6\": container with ID starting with 1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6 not found: ID does not exist" containerID="1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6"
Sep 11 18:45:38.469308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.469229    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6"} err="failed to get container status \"1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6\": rpc error: code = NotFound desc = could not find container \"1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6\": container with ID starting with 1c855d6b86c3025349b5a7ac854598f163ec200c53a49afb462573b0c02172c6 not found: ID does not exist"
Sep 11 18:45:38.469308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.469245    2563 scope.go:117] "RemoveContainer" containerID="58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"
Sep 11 18:45:38.469559 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:45:38.469541    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35\": container with ID starting with 58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35 not found: ID does not exist" containerID="58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"
Sep 11 18:45:38.469600 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.469563    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35"} err="failed to get container status \"58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35\": rpc error: code = NotFound desc = could not find container \"58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35\": container with ID starting with 58cf887cc2c14bb295f5830d69cfa888af06c5edaa56f8fd4b6c9a6a4f3a0c35 not found: ID does not exist"
Sep 11 18:45:38.472535 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.472512    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"]
Sep 11 18:45:38.476024 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:38.476004    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-5faf38-openshift-default-855bf88d44-rxslw"]
Sep 11 18:45:39.911947 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:45:39.911913    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" path="/var/lib/kubelet/pods/8f0702ba-9a7e-452f-a7d4-a363bdbd2c66/volumes"
Sep 11 18:46:36.312287 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312241    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-556697db7-j2sp9"]
Sep 11 18:46:36.312696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312668    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy"
Sep 11 18:46:36.312696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312681    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc8e855-11a4-4ffc-b116-6b06aa1cfd40" containerName="istio-proxy"
Sep 11 18:46:36.312696 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312695    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312702    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312716    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312722    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d76030a9-5f07-46a5-a53a-44934691be4a" containerName="cleanup"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312728    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d76030a9-5f07-46a5-a53a-44934691be4a" containerName="cleanup"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312736    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" containerName="payload-processing"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312741    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" containerName="payload-processing"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312776    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71633f76-16bd-4065-8521-af0bdb28a5d1" containerName="manager"
Sep 11 18:46:36.312821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312781    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71633f76-16bd-4065-8521-af0bdb28a5d1" containerName="manager"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312848    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312855    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312860    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d76030a9-5f07-46a5-a53a-44934691be4a" containerName="cleanup"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312867    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312875    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8833bc1b-04fb-4f06-9af4-6b6ff28e4c99" containerName="payload-processing"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312881    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71633f76-16bd-4065-8521-af0bdb28a5d1" containerName="manager"
Sep 11 18:46:36.313081 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.312886    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy"
Sep 11 18:46:36.316845 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.316825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.323531 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.323509    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 18:46:36.323625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.323552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 18:46:36.323625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.323510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mh8vn\""
Sep 11 18:46:36.323625 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.323587    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 18:46:36.337354 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.337332    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-556697db7-j2sp9"]
Sep 11 18:46:36.372335 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.372310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-metrics-certs\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.372457 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.372348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvbxd\" (UniqueName: \"kubernetes.io/projected/881d1d7f-0be5-4b81-848d-0af6799a231e-kube-api-access-cvbxd\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.372533 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.372509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-webhook-cert\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.473581 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.473551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-webhook-cert\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.473720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.473588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-metrics-certs\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.473720 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.473617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvbxd\" (UniqueName: \"kubernetes.io/projected/881d1d7f-0be5-4b81-848d-0af6799a231e-kube-api-access-cvbxd\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.476108 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.476081    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-metrics-certs\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.476198 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.476123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-webhook-cert\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.483152 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.483131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvbxd\" (UniqueName: \"kubernetes.io/projected/881d1d7f-0be5-4b81-848d-0af6799a231e-kube-api-access-cvbxd\") pod \"maas-controller-556697db7-j2sp9\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") " pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.626695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.626629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:36.901592 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.901564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-556697db7-j2sp9"]
Sep 11 18:46:36.902901 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:46:36.902877    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881d1d7f_0be5_4b81_848d_0af6799a231e.slice/crio-a7764598f62e27236b0c202e7df2469101defe5b6f5a316d8447711b82bf4384 WatchSource:0}: Error finding container a7764598f62e27236b0c202e7df2469101defe5b6f5a316d8447711b82bf4384: Status 404 returned error can't find the container with id a7764598f62e27236b0c202e7df2469101defe5b6f5a316d8447711b82bf4384
Sep 11 18:46:36.904211 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:36.904193    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:46:37.755485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:37.755400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-j2sp9" event={"ID":"881d1d7f-0be5-4b81-848d-0af6799a231e","Type":"ContainerStarted","Data":"cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a"}
Sep 11 18:46:37.755485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:37.755451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-j2sp9" event={"ID":"881d1d7f-0be5-4b81-848d-0af6799a231e","Type":"ContainerStarted","Data":"a7764598f62e27236b0c202e7df2469101defe5b6f5a316d8447711b82bf4384"}
Sep 11 18:46:37.755861 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:37.755509    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:37.775235 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:37.775171    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-556697db7-j2sp9" podStartSLOduration=1.31660735 podStartE2EDuration="1.775158514s" totalImagesPullingTime="458.551164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:46:37.774669417 +0000 UTC m=+2166.367869974" watchObservedRunningTime="2026-09-11 18:46:37.775158514 +0000 UTC m=+2166.368359072"
Sep 11 18:46:48.763214 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:48.763184    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:55.897884 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:55.897854    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:46:55.999250 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:55.999215    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-556697db7-j2sp9"]
Sep 11 18:46:55.999476 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:55.999451    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-556697db7-j2sp9" podUID="881d1d7f-0be5-4b81-848d-0af6799a231e" containerName="manager" containerID="cri-o://cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a" gracePeriod=30
Sep 11 18:46:56.244571 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.244548    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:56.342812 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.342783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-metrics-certs\") pod \"881d1d7f-0be5-4b81-848d-0af6799a231e\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") "
Sep 11 18:46:56.342812 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.342819    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvbxd\" (UniqueName: \"kubernetes.io/projected/881d1d7f-0be5-4b81-848d-0af6799a231e-kube-api-access-cvbxd\") pod \"881d1d7f-0be5-4b81-848d-0af6799a231e\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") "
Sep 11 18:46:56.342998 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.342850    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-webhook-cert\") pod \"881d1d7f-0be5-4b81-848d-0af6799a231e\" (UID: \"881d1d7f-0be5-4b81-848d-0af6799a231e\") "
Sep 11 18:46:56.344821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.344790    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "881d1d7f-0be5-4b81-848d-0af6799a231e" (UID: "881d1d7f-0be5-4b81-848d-0af6799a231e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:46:56.344958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.344840    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "881d1d7f-0be5-4b81-848d-0af6799a231e" (UID: "881d1d7f-0be5-4b81-848d-0af6799a231e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:46:56.344958 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.344843    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881d1d7f-0be5-4b81-848d-0af6799a231e-kube-api-access-cvbxd" (OuterVolumeSpecName: "kube-api-access-cvbxd") pod "881d1d7f-0be5-4b81-848d-0af6799a231e" (UID: "881d1d7f-0be5-4b81-848d-0af6799a231e"). InnerVolumeSpecName "kube-api-access-cvbxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:46:56.444001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.443944    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:46:56.444001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.443965    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvbxd\" (UniqueName: \"kubernetes.io/projected/881d1d7f-0be5-4b81-848d-0af6799a231e-kube-api-access-cvbxd\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:46:56.444001 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.443974    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/881d1d7f-0be5-4b81-848d-0af6799a231e-webhook-cert\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:46:56.851979 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.851938    2563 generic.go:356] "Generic (PLEG): container finished" podID="881d1d7f-0be5-4b81-848d-0af6799a231e" containerID="cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a" exitCode=0
Sep 11 18:46:56.852186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.851994    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-j2sp9"
Sep 11 18:46:56.852186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.852006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-j2sp9" event={"ID":"881d1d7f-0be5-4b81-848d-0af6799a231e","Type":"ContainerDied","Data":"cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a"}
Sep 11 18:46:56.852186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.852058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-j2sp9" event={"ID":"881d1d7f-0be5-4b81-848d-0af6799a231e","Type":"ContainerDied","Data":"a7764598f62e27236b0c202e7df2469101defe5b6f5a316d8447711b82bf4384"}
Sep 11 18:46:56.852186 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.852088    2563 scope.go:117] "RemoveContainer" containerID="cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a"
Sep 11 18:46:56.864478 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.864457    2563 scope.go:117] "RemoveContainer" containerID="cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a"
Sep 11 18:46:56.864786 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:46:56.864756    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a\": container with ID starting with cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a not found: ID does not exist" containerID="cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a"
Sep 11 18:46:56.864889 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.864797    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a"} err="failed to get container status \"cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a\": rpc error: code = NotFound desc = could not find container \"cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a\": container with ID starting with cdfa571f66e43a1bc0c82dcf86ea5cfef2b611874086fc9fb11daf07d96c783a not found: ID does not exist"
Sep 11 18:46:56.874791 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.874767    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-556697db7-j2sp9"]
Sep 11 18:46:56.878410 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:56.878389    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-556697db7-j2sp9"]
Sep 11 18:46:57.557202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.557155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:46:57.677431 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677399    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-556697db7-q5mx7"]
Sep 11 18:46:57.677821 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677807    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="881d1d7f-0be5-4b81-848d-0af6799a231e" containerName="manager"
Sep 11 18:46:57.677865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677823    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="881d1d7f-0be5-4b81-848d-0af6799a231e" containerName="manager"
Sep 11 18:46:57.677865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677840    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy"
Sep 11 18:46:57.677865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677846    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="51468e39-25ed-4574-9768-3fe097f31a53" containerName="istio-proxy"
Sep 11 18:46:57.677865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677853    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy"
Sep 11 18:46:57.677865 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677858    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f0702ba-9a7e-452f-a7d4-a363bdbd2c66" containerName="istio-proxy"
Sep 11 18:46:57.678017 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.677915    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="881d1d7f-0be5-4b81-848d-0af6799a231e" containerName="manager"
Sep 11 18:46:57.682330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.682310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.685974 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.685940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 18:46:57.686086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.685986    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 18:46:57.686086 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.686068    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 18:46:57.686253 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.686236    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mh8vn\""
Sep 11 18:46:57.701005 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.700979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-556697db7-q5mx7"]
Sep 11 18:46:57.755297 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.755248    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a27d8ea-8824-436a-b4b0-e61dee103e46-webhook-cert\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.755390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.755314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a27d8ea-8824-436a-b4b0-e61dee103e46-metrics-certs\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.755390 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.755363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plctq\" (UniqueName: \"kubernetes.io/projected/3a27d8ea-8824-436a-b4b0-e61dee103e46-kube-api-access-plctq\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.856733 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.856669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a27d8ea-8824-436a-b4b0-e61dee103e46-webhook-cert\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.856733 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.856726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a27d8ea-8824-436a-b4b0-e61dee103e46-metrics-certs\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.856887 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.856764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plctq\" (UniqueName: \"kubernetes.io/projected/3a27d8ea-8824-436a-b4b0-e61dee103e46-kube-api-access-plctq\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.859087 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.859067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3a27d8ea-8824-436a-b4b0-e61dee103e46-webhook-cert\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.859175 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.859068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a27d8ea-8824-436a-b4b0-e61dee103e46-metrics-certs\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.864891 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.864869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plctq\" (UniqueName: \"kubernetes.io/projected/3a27d8ea-8824-436a-b4b0-e61dee103e46-kube-api-access-plctq\") pod \"maas-controller-556697db7-q5mx7\" (UID: \"3a27d8ea-8824-436a-b4b0-e61dee103e46\") " pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:57.912110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.912086    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="881d1d7f-0be5-4b81-848d-0af6799a231e" path="/var/lib/kubelet/pods/881d1d7f-0be5-4b81-848d-0af6799a231e/volumes"
Sep 11 18:46:57.992668 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:57.992643    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:58.266113 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:58.266036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-556697db7-q5mx7"]
Sep 11 18:46:58.269178 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:46:58.269152    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a27d8ea_8824_436a_b4b0_e61dee103e46.slice/crio-5f78a33edb9ef701017de74bae61dd64dddffee962c0b76c86c1ef192e5ed650 WatchSource:0}: Error finding container 5f78a33edb9ef701017de74bae61dd64dddffee962c0b76c86c1ef192e5ed650: Status 404 returned error can't find the container with id 5f78a33edb9ef701017de74bae61dd64dddffee962c0b76c86c1ef192e5ed650
Sep 11 18:46:58.865861 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:58.865773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-q5mx7" event={"ID":"3a27d8ea-8824-436a-b4b0-e61dee103e46","Type":"ContainerStarted","Data":"bdfb7abc9402d482c42a4e47da9a789ff1528bfa7364ed6c2ce1cd6963e475e2"}
Sep 11 18:46:58.865861 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:58.865841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-556697db7-q5mx7" event={"ID":"3a27d8ea-8824-436a-b4b0-e61dee103e46","Type":"ContainerStarted","Data":"5f78a33edb9ef701017de74bae61dd64dddffee962c0b76c86c1ef192e5ed650"}
Sep 11 18:46:58.866232 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:58.865910    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:46:58.884880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:46:58.884839    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-556697db7-q5mx7" podStartSLOduration=1.565124403 podStartE2EDuration="1.884825599s" totalImagesPullingTime="319.701196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:46:58.883597146 +0000 UTC m=+2187.476797704" watchObservedRunningTime="2026-09-11 18:46:58.884825599 +0000 UTC m=+2187.478026156"
Sep 11 18:47:09.873475 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:47:09.873441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-556697db7-q5mx7"
Sep 11 18:47:25.300809 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:47:25.300773    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:47:28.446946 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:47:28.446914    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:49:09.944075 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:49:09.944035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:49:21.475308 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:49:21.475251    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:49:21.528382 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:49:21.528349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:49:45.537034 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:49:45.537001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:49:51.913110 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:49:51.913055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:50:04.633507 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:04.633464    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:50:04.734187 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:04.734161    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:50:30.963022 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:30.962987    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:50:32.083724 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:32.083698    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:50:32.093408 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:32.093380    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7bpkd_a11915f1-cc9f-46ab-b2de-44c8aa7a2e86/ovn-acl-logging/0.log"
Sep 11 18:50:32.418117 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:32.418043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:50:33.184143 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:33.184106    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:50:41.290127 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:50:41.290087    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:51:05.486672 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:05.486596    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"]
Sep 11 18:51:05.487191 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:05.486819    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf" podUID="87912f1f-7401-434e-9e96-24ed9db35d12" containerName="maas-api" containerID="cri-o://9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729" gracePeriod=30
Sep 11 18:51:06.738043 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.738014    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:51:06.778599 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.778569    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87912f1f-7401-434e-9e96-24ed9db35d12-openshift-service-ca\") pod \"87912f1f-7401-434e-9e96-24ed9db35d12\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") "
Sep 11 18:51:06.778750 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.778605    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjx49\" (UniqueName: \"kubernetes.io/projected/87912f1f-7401-434e-9e96-24ed9db35d12-kube-api-access-wjx49\") pod \"87912f1f-7401-434e-9e96-24ed9db35d12\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") "
Sep 11 18:51:06.778807 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.778789    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs\") pod \"87912f1f-7401-434e-9e96-24ed9db35d12\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") "
Sep 11 18:51:06.778875 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.778861    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls\") pod \"87912f1f-7401-434e-9e96-24ed9db35d12\" (UID: \"87912f1f-7401-434e-9e96-24ed9db35d12\") "
Sep 11 18:51:06.778932 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.778903    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87912f1f-7401-434e-9e96-24ed9db35d12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87912f1f-7401-434e-9e96-24ed9db35d12" (UID: "87912f1f-7401-434e-9e96-24ed9db35d12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:51:06.779298 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.779272    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87912f1f-7401-434e-9e96-24ed9db35d12-openshift-service-ca\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:51:06.780793 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.780765    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "87912f1f-7401-434e-9e96-24ed9db35d12" (UID: "87912f1f-7401-434e-9e96-24ed9db35d12"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:51:06.781164 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.781137    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87912f1f-7401-434e-9e96-24ed9db35d12" (UID: "87912f1f-7401-434e-9e96-24ed9db35d12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:51:06.781217 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.781141    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87912f1f-7401-434e-9e96-24ed9db35d12-kube-api-access-wjx49" (OuterVolumeSpecName: "kube-api-access-wjx49") pod "87912f1f-7401-434e-9e96-24ed9db35d12" (UID: "87912f1f-7401-434e-9e96-24ed9db35d12"). InnerVolumeSpecName "kube-api-access-wjx49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:51:06.879922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.879860    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-maas-api-tls\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:51:06.879922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.879895    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjx49\" (UniqueName: \"kubernetes.io/projected/87912f1f-7401-434e-9e96-24ed9db35d12-kube-api-access-wjx49\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:51:06.879922 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:06.879911    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87912f1f-7401-434e-9e96-24ed9db35d12-metrics-certs\") on node \"ip-10-0-140-142.ec2.internal\" DevicePath \"\""
Sep 11 18:51:07.073708 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.073675    2563 generic.go:356] "Generic (PLEG): container finished" podID="87912f1f-7401-434e-9e96-24ed9db35d12" containerID="9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729" exitCode=0
Sep 11 18:51:07.073880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.073717    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"
Sep 11 18:51:07.073880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.073756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf" event={"ID":"87912f1f-7401-434e-9e96-24ed9db35d12","Type":"ContainerDied","Data":"9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729"}
Sep 11 18:51:07.073880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.073804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf" event={"ID":"87912f1f-7401-434e-9e96-24ed9db35d12","Type":"ContainerDied","Data":"e0603309ba3b5c6b54974b7f97d8337e9fe829d6500e03dd236a05d15e112bc5"}
Sep 11 18:51:07.073880 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.073834    2563 scope.go:117] "RemoveContainer" containerID="9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729"
Sep 11 18:51:07.083731 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.083715    2563 scope.go:117] "RemoveContainer" containerID="9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729"
Sep 11 18:51:07.084035 ip-10-0-140-142 kubenswrapper[2563]: E0911 18:51:07.084018    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729\": container with ID starting with 9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729 not found: ID does not exist" containerID="9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729"
Sep 11 18:51:07.084101 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.084049    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729"} err="failed to get container status \"9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729\": rpc error: code = NotFound desc = could not find container \"9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729\": container with ID starting with 9db96999c73aee883c0f683a034175e0b0a3ed3580d654b1c092a8b8ee0b3729 not found: ID does not exist"
Sep 11 18:51:07.111161 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.111136    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"]
Sep 11 18:51:07.116545 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.116522    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-56d545b69f-vg8sf"]
Sep 11 18:51:07.912613 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:07.912571    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87912f1f-7401-434e-9e96-24ed9db35d12" path="/var/lib/kubelet/pods/87912f1f-7401-434e-9e96-24ed9db35d12/volumes"
Sep 11 18:51:20.059523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:20.059484    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9576b5555-k69zb_5c5d0def-fccd-40ec-ac33-31898ff4f6bf/authorino/0.log"
Sep 11 18:51:25.510788 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:25.510738    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-556697db7-q5mx7_3a27d8ea-8824-436a-b4b0-e61dee103e46/manager/0.log"
Sep 11 18:51:26.074604 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:26.074577    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657fcdcb9c-nrzn2_b4972f6f-57c6-4d5d-957a-753453be6f05/copy-manifests/0.log"
Sep 11 18:51:26.081097 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:26.081074    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657fcdcb9c-nrzn2_b4972f6f-57c6-4d5d-957a-753453be6f05/manager/0.log"
Sep 11 18:51:27.615213 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:27.615182    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9576b5555-k69zb_5c5d0def-fccd-40ec-ac33-31898ff4f6bf/authorino/0.log"
Sep 11 18:51:28.973281 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:28.973212    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b485767cb-wgfwb_465c25cf-1958-4d11-9ab4-d6728ebcd361/kube-auth-proxy/0.log"
Sep 11 18:51:29.086938 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:29.086904    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/1.log"
Sep 11 18:51:29.318513 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:29.318483    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-gc5jg_d1aeb3a3-97e1-4310-a598-0346c7e1231b/payload-processing/0.log"
Sep 11 18:51:29.856644 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:29.856610    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gr95g_703963e3-d628-4e2d-8521-3bcde98f6020/main/0.log"
Sep 11 18:51:30.182434 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:30.182349    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frsdp6_c5e5f941-2b3a-4764-a35c-b68d89773a76/main/0.log"
Sep 11 18:51:30.406725 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:30.406692    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-xdk2p_857309fa-3fa4-4717-8c29-6e69febe291c/main/0.log"
Sep 11 18:51:32.341760 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:32.341728    2563 scope.go:117] "RemoveContainer" containerID="84cb1453e17ac6c20d2a1b15aca3bbf1c24ae25f9668f32fbdd13c18a0fb3e6d"
Sep 11 18:51:32.350028 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:32.350003    2563 scope.go:117] "RemoveContainer" containerID="f7d30994e9495d95b1f42bdf22425e5ce6619984f8e752bced79401500d5ae52"
Sep 11 18:51:35.662760 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.662720    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8rt2b/must-gather-4zplf"]
Sep 11 18:51:35.663309 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.663289    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87912f1f-7401-434e-9e96-24ed9db35d12" containerName="maas-api"
Sep 11 18:51:35.663409 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.663312    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="87912f1f-7401-434e-9e96-24ed9db35d12" containerName="maas-api"
Sep 11 18:51:35.663468 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.663451    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="87912f1f-7401-434e-9e96-24ed9db35d12" containerName="maas-api"
Sep 11 18:51:35.668518 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.668494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.671181 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.671155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8rt2b\"/\"default-dockercfg-65ww9\""
Sep 11 18:51:35.671330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.671198    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8rt2b\"/\"kube-root-ca.crt\""
Sep 11 18:51:35.671330 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.671195    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8rt2b\"/\"openshift-service-ca.crt\""
Sep 11 18:51:35.678370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.678347    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8rt2b/must-gather-4zplf"]
Sep 11 18:51:35.680774 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.680749    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:51:35.782511 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.782485    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log" limit=1024
Sep 11 18:51:35.847305 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.847277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1e6bc93-f6bf-4eac-8046-b13a82d67689-must-gather-output\") pod \"must-gather-4zplf\" (UID: \"f1e6bc93-f6bf-4eac-8046-b13a82d67689\") " pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.847397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.847348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tmf8\" (UniqueName: \"kubernetes.io/projected/f1e6bc93-f6bf-4eac-8046-b13a82d67689-kube-api-access-8tmf8\") pod \"must-gather-4zplf\" (UID: \"f1e6bc93-f6bf-4eac-8046-b13a82d67689\") " pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.948357 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.948305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1e6bc93-f6bf-4eac-8046-b13a82d67689-must-gather-output\") pod \"must-gather-4zplf\" (UID: \"f1e6bc93-f6bf-4eac-8046-b13a82d67689\") " pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.948357 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.948350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tmf8\" (UniqueName: \"kubernetes.io/projected/f1e6bc93-f6bf-4eac-8046-b13a82d67689-kube-api-access-8tmf8\") pod \"must-gather-4zplf\" (UID: \"f1e6bc93-f6bf-4eac-8046-b13a82d67689\") " pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.948620 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.948595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1e6bc93-f6bf-4eac-8046-b13a82d67689-must-gather-output\") pod \"must-gather-4zplf\" (UID: \"f1e6bc93-f6bf-4eac-8046-b13a82d67689\") " pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.955529 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.955506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tmf8\" (UniqueName: \"kubernetes.io/projected/f1e6bc93-f6bf-4eac-8046-b13a82d67689-kube-api-access-8tmf8\") pod \"must-gather-4zplf\" (UID: \"f1e6bc93-f6bf-4eac-8046-b13a82d67689\") " pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:35.978544 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:35.978527    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8rt2b/must-gather-4zplf"
Sep 11 18:51:36.265115 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:36.265091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8rt2b/must-gather-4zplf"]
Sep 11 18:51:36.267215 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:51:36.267185    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1e6bc93_f6bf_4eac_8046_b13a82d67689.slice/crio-2761398536cb1f453c90d98d2d81569a346c039797f6a77465434c50240b11cb WatchSource:0}: Error finding container 2761398536cb1f453c90d98d2d81569a346c039797f6a77465434c50240b11cb: Status 404 returned error can't find the container with id 2761398536cb1f453c90d98d2d81569a346c039797f6a77465434c50240b11cb
Sep 11 18:51:37.232829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:37.232751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rt2b/must-gather-4zplf" event={"ID":"f1e6bc93-f6bf-4eac-8046-b13a82d67689","Type":"ContainerStarted","Data":"1cb3c5f65a6d40b0aec57bdc2b891bfa5518533835d83c0735f79bb3f58f8667"}
Sep 11 18:51:37.232829 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:37.232805    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rt2b/must-gather-4zplf" event={"ID":"f1e6bc93-f6bf-4eac-8046-b13a82d67689","Type":"ContainerStarted","Data":"2761398536cb1f453c90d98d2d81569a346c039797f6a77465434c50240b11cb"}
Sep 11 18:51:38.240190 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:38.240155    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rt2b/must-gather-4zplf" event={"ID":"f1e6bc93-f6bf-4eac-8046-b13a82d67689","Type":"ContainerStarted","Data":"81ffc6836fb8dd53983a804a202bd5ac007f2436786263da45cc061463915578"}
Sep 11 18:51:38.258114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:38.258059    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8rt2b/must-gather-4zplf" podStartSLOduration=2.459433945 podStartE2EDuration="3.258039218s" totalImagesPullingTime="798.605273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:38.254791237 +0000 UTC m=+2466.847991792" watchObservedRunningTime="2026-09-11 18:51:38.258039218 +0000 UTC m=+2466.851239776"
Sep 11 18:51:38.703050 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:38.703012    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6ntmn_5f2089a3-9817-4c29-980b-f4018a7b145e/global-pull-secret-syncer/0.log"
Sep 11 18:51:38.843101 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:38.843066    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4vhkv_c3e22d68-3089-47b8-b9b1-1c81db256a73/konnectivity-agent/0.log"
Sep 11 18:51:38.983523 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:38.983444    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-142.ec2.internal_9add31ed36a5b5cac8ed65d2c77f17bf/haproxy/0.log"
Sep 11 18:51:43.225282 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:43.224718    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9576b5555-k69zb_5c5d0def-fccd-40ec-ac33-31898ff4f6bf/authorino/0.log"
Sep 11 18:51:45.168977 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.168937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-6hwgd_05b2274a-8be6-4170-9723-81ff3601f471/cluster-monitoring-operator/0.log"
Sep 11 18:51:45.200637 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.200592    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-6xlf9_24ef2858-8dbc-4a5f-97f7-fe8cc5702fab/kube-state-metrics/0.log"
Sep 11 18:51:45.223164 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.223131    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-6xlf9_24ef2858-8dbc-4a5f-97f7-fe8cc5702fab/kube-rbac-proxy-main/0.log"
Sep 11 18:51:45.248810 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.248778    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-6xlf9_24ef2858-8dbc-4a5f-97f7-fe8cc5702fab/kube-rbac-proxy-self/0.log"
Sep 11 18:51:45.510463 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.510382    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zmg88_3ccc46eb-455f-4156-bab2-cd071622b409/node-exporter/0.log"
Sep 11 18:51:45.533175 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.533088    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zmg88_3ccc46eb-455f-4156-bab2-cd071622b409/kube-rbac-proxy/0.log"
Sep 11 18:51:45.563780 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.563747    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zmg88_3ccc46eb-455f-4156-bab2-cd071622b409/init-textfile/0.log"
Sep 11 18:51:45.606673 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.606568    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-n9wwr_a3c07754-c93c-4a26-a5c7-d8484b9bc358/kube-rbac-proxy-main/0.log"
Sep 11 18:51:45.633941 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.633839    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-n9wwr_a3c07754-c93c-4a26-a5c7-d8484b9bc358/kube-rbac-proxy-self/0.log"
Sep 11 18:51:45.668133 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.668100    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-n9wwr_a3c07754-c93c-4a26-a5c7-d8484b9bc358/openshift-state-metrics/0.log"
Sep 11 18:51:45.717058 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.716661    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/prometheus/0.log"
Sep 11 18:51:45.743275 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.743216    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/config-reloader/0.log"
Sep 11 18:51:45.769114 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.769037    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/thanos-sidecar/0.log"
Sep 11 18:51:45.798734 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.798697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/kube-rbac-proxy-web/0.log"
Sep 11 18:51:45.829556 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.829522    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/kube-rbac-proxy/0.log"
Sep 11 18:51:45.852737 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.852711    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/kube-rbac-proxy-thanos/0.log"
Sep 11 18:51:45.879045 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:45.879015    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa650c97-9230-4c35-a6d8-e81a3975ade2/init-config-reloader/0.log"
Sep 11 18:51:46.025927 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.025897    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66c5794976-gwqk2_58a9d0a8-0153-47bd-8437-b0e7d2506a4a/telemeter-client/0.log"
Sep 11 18:51:46.054332 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.053631    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66c5794976-gwqk2_58a9d0a8-0153-47bd-8437-b0e7d2506a4a/reload/0.log"
Sep 11 18:51:46.090583 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.090496    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66c5794976-gwqk2_58a9d0a8-0153-47bd-8437-b0e7d2506a4a/kube-rbac-proxy/0.log"
Sep 11 18:51:46.131065 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.130087    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655bb94d9f-62jf9_3a9bfd69-36cd-4858-8647-7b358c4f9a1c/thanos-query/0.log"
Sep 11 18:51:46.155588 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.155435    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655bb94d9f-62jf9_3a9bfd69-36cd-4858-8647-7b358c4f9a1c/kube-rbac-proxy-web/0.log"
Sep 11 18:51:46.179641 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.179611    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655bb94d9f-62jf9_3a9bfd69-36cd-4858-8647-7b358c4f9a1c/kube-rbac-proxy/0.log"
Sep 11 18:51:46.207138 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.207111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655bb94d9f-62jf9_3a9bfd69-36cd-4858-8647-7b358c4f9a1c/prom-label-proxy/0.log"
Sep 11 18:51:46.231506 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.231475    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655bb94d9f-62jf9_3a9bfd69-36cd-4858-8647-7b358c4f9a1c/kube-rbac-proxy-rules/0.log"
Sep 11 18:51:46.256469 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:46.256428    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655bb94d9f-62jf9_3a9bfd69-36cd-4858-8647-7b358c4f9a1c/kube-rbac-proxy-metrics/0.log"
Sep 11 18:51:48.885814 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:48.885763    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"]
Sep 11 18:51:48.893370 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:48.893340    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:48.899351 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:48.899242    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"]
Sep 11 18:51:49.007057 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.007006    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-lib-modules\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.007249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.007110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdxgb\" (UniqueName: \"kubernetes.io/projected/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-kube-api-access-fdxgb\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.007249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.007158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-proc\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.007249 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.007208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-sys\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.007453 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.007332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-podres\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108076 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-proc\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-sys\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-proc\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-podres\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108182    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-sys\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108242 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-lib-modules\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdxgb\" (UniqueName: \"kubernetes.io/projected/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-kube-api-access-fdxgb\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108305    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-podres\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.108525 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.108427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-lib-modules\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.117386 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.117358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdxgb\" (UniqueName: \"kubernetes.io/projected/e5e39c9e-cd35-46ff-b0d6-5246e8375c17-kube-api-access-fdxgb\") pod \"perf-node-gather-daemonset-qn427\" (UID: \"e5e39c9e-cd35-46ff-b0d6-5246e8375c17\") " pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.212658 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.212564    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:49.505440 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.505356    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"]
Sep 11 18:51:49.509592 ip-10-0-140-142 kubenswrapper[2563]: W0911 18:51:49.509560    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode5e39c9e_cd35_46ff_b0d6_5246e8375c17.slice/crio-5368977907e4aeaa0f6e211d877fec35f2759a85ac3393e04b383decfbfad8f5 WatchSource:0}: Error finding container 5368977907e4aeaa0f6e211d877fec35f2759a85ac3393e04b383decfbfad8f5: Status 404 returned error can't find the container with id 5368977907e4aeaa0f6e211d877fec35f2759a85ac3393e04b383decfbfad8f5
Sep 11 18:51:49.955397 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:49.955361    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-vtq6n_72ba54c6-8c8f-4fba-9954-b6e3cd3f9fb2/volume-data-source-validator/0.log"
Sep 11 18:51:50.346035 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:50.345824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427" event={"ID":"e5e39c9e-cd35-46ff-b0d6-5246e8375c17","Type":"ContainerStarted","Data":"3cc49849c450d824017ba38afc94f7c719c3b041b6cdd74996da9b17fdf20094"}
Sep 11 18:51:50.346225 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:50.346159    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:50.346225 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:50.346188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427" event={"ID":"e5e39c9e-cd35-46ff-b0d6-5246e8375c17","Type":"ContainerStarted","Data":"5368977907e4aeaa0f6e211d877fec35f2759a85ac3393e04b383decfbfad8f5"}
Sep 11 18:51:50.364230 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:50.364178    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427" podStartSLOduration=2.36415869 podStartE2EDuration="2.36415869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:51:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:51:50.360956388 +0000 UTC m=+2478.954156946" watchObservedRunningTime="2026-09-11 18:51:50.36415869 +0000 UTC m=+2478.957359248"
Sep 11 18:51:51.094231 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:51.094199    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vk549_6b50bc98-cf61-4994-bcfd-9c43af9229e4/dns/0.log"
Sep 11 18:51:51.118199 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:51.118172    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vk549_6b50bc98-cf61-4994-bcfd-9c43af9229e4/kube-rbac-proxy/0.log"
Sep 11 18:51:51.254068 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:51.254038    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x46v2_3b656de9-f949-4ae4-b1a7-9ee77032b734/dns-node-resolver/0.log"
Sep 11 18:51:51.792247 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:51.792218    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-db649f896-wzjc8_37c05e10-e9f0-4f5b-beee-8617ab2ffc96/registry/0.log"
Sep 11 18:51:51.815171 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:51.815129    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4ft7z_117de80d-dc8d-43d8-b406-85c3a636ecb4/node-ca/0.log"
Sep 11 18:51:53.274627 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:53.274602    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b485767cb-wgfwb_465c25cf-1958-4d11-9ab4-d6728ebcd361/kube-auth-proxy/0.log"
Sep 11 18:51:53.307866 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:53.307830    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/0.log"
Sep 11 18:51:53.310992 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:53.310970    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/1.log"
Sep 11 18:51:53.387651 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:53.387621    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-gc5jg_d1aeb3a3-97e1-4310-a598-0346c7e1231b/payload-processing/0.log"
Sep 11 18:51:54.389111 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:54.389079    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rjkqn_0266d274-0277-49ea-ac36-9e8bf474fae3/serve-healthcheck-canary/0.log"
Sep 11 18:51:54.924169 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:54.924133    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-jkm2r_8035ce1c-4e18-4f71-bbd7-65643b813d0e/insights-operator/0.log"
Sep 11 18:51:54.924800 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:54.924771    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-jkm2r_8035ce1c-4e18-4f71-bbd7-65643b813d0e/insights-operator/1.log"
Sep 11 18:51:54.945107 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:54.945068    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8v4wp_006f2d67-3840-4a1b-b6b8-7ead931c73c2/kube-rbac-proxy/0.log"
Sep 11 18:51:54.970712 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:54.970688    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8v4wp_006f2d67-3840-4a1b-b6b8-7ead931c73c2/exporter/0.log"
Sep 11 18:51:54.994202 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:54.994177    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8v4wp_006f2d67-3840-4a1b-b6b8-7ead931c73c2/extractor/0.log"
Sep 11 18:51:56.363533 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:56.363504    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8rt2b/perf-node-gather-daemonset-qn427"
Sep 11 18:51:58.009144 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:58.009100    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-556697db7-q5mx7_3a27d8ea-8824-436a-b4b0-e61dee103e46/manager/0.log"
Sep 11 18:51:58.359072 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:58.359042    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657fcdcb9c-nrzn2_b4972f6f-57c6-4d5d-957a-753453be6f05/manager/0.log"
Sep 11 18:51:58.382679 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:58.382654    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657fcdcb9c-nrzn2_b4972f6f-57c6-4d5d-957a-753453be6f05/copy-manifests/0.log"
Sep 11 18:51:59.375633 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:59.375562    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b485767cb-wgfwb_465c25cf-1958-4d11-9ab4-d6728ebcd361/kube-auth-proxy/0.log"
Sep 11 18:51:59.395695 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:59.395665    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/0.log"
Sep 11 18:51:59.402104 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:59.402060    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/1.log"
Sep 11 18:51:59.457414 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:59.457381    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-gc5jg_d1aeb3a3-97e1-4310-a598-0346c7e1231b/payload-processing/0.log"
Sep 11 18:51:59.760392 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:51:59.760310    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6dd5f9dbb5-m69w6_253e6e7d-d83c-4d5e-9c24-9fc1e8aa28a8/manager/0.log"
Sep 11 18:52:01.429730 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:01.429662    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b485767cb-wgfwb_465c25cf-1958-4d11-9ab4-d6728ebcd361/kube-auth-proxy/0.log"
Sep 11 18:52:01.452253 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:01.452224    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/0.log"
Sep 11 18:52:01.457006 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:01.456980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-2gjdw_ed068df9-4d54-4e39-90a8-933b710e11c0/istio-proxy/1.log"
Sep 11 18:52:01.501425 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:01.501348    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-gc5jg_d1aeb3a3-97e1-4310-a598-0346c7e1231b/payload-processing/0.log"
Sep 11 18:52:05.258355 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:05.258322    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-mbn7j_f6687a09-2cda-40ca-8023-71593b98bf54/kube-storage-version-migrator-operator/1.log"
Sep 11 18:52:05.260144 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:05.260120    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-mbn7j_f6687a09-2cda-40ca-8023-71593b98bf54/kube-storage-version-migrator-operator/0.log"
Sep 11 18:52:06.274685 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.274608    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9b54z_06168f80-3164-4c7e-a316-f7449b680db9/kube-multus/0.log"
Sep 11 18:52:06.295945 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.295923    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/kube-multus-additional-cni-plugins/0.log"
Sep 11 18:52:06.324492 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.324473    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/egress-router-binary-copy/0.log"
Sep 11 18:52:06.345325 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.345309    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/cni-plugins/0.log"
Sep 11 18:52:06.367835 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.367817    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/bond-cni-plugin/0.log"
Sep 11 18:52:06.388171 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.388144    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/routeoverride-cni/0.log"
Sep 11 18:52:06.409485 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.409464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/whereabouts-cni-bincopy/0.log"
Sep 11 18:52:06.433550 ip-10-0-140-142 kubenswrapper[2563]: I0911 18:52:06.433524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4rdld_a3a338ce-174e-464b-8a7a-7fee4c5dd1ee/whereabouts-cni/0.log"